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1. SUMMARY 

Concord Scientific Corporation (CSC) has been contracted by 
the Ontario Ministry of the Environment (MOE) to carry out two audits of 
the Acidic Precipitation in Ontario Study CAPIOS) monitoring network 
(laboratory activities were not included in the audit since these 
aspects were extensively addressed in a previous audit, (CSC, 1982)). 

This quality assurance audit provides an independent review 
and evaluation of the operating and quality assurance procedures for the 
APIOS monitoring network. As defined in Phase I, the objective of the 
audit is to assess how the various components within the APIOS programme 
are contributing to the production of "high quality" data. 

In general, the Phase II audits indicate that the APIOS 
personnel are properly executing the procedures that are documented in 
the APIOS Technical and Operating Manual (Bardswick, 1983). 

Operators' training for the most part has been upgraded or 
reinforced and operators are following the currently recommended opera- 
ting and handling procedures. This ensures that good quality data are 
collected. 

The majority of the monitoring sites visited during this audit 
met siting criteria and the instruments were operating according to 
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design specifications. Sites which are in violation of the siting 
criteria, or operators not using approved handling procedures and there- 
fore jeopardizing sample integrity, are described in detail in the 
performance related findings of the audit report {Section 3.1). A 
special review by the QA Coordinator (QAC) of these specific sites and 
operators should be carried out. 

Since the Phase I audit, the data validation procedures for 
cumulative and daily air concentration data have been documented (Chung, 
1984; Kirk, 1984) and the 1982 daily and cumulative air concentration 
listings have been published (MOE, 1984). These data validation proce- 
dures have been well conceived in general, however, some recommendations 
with reference to data qualification and reporting have been made 
(Section 4.1). 

As recommended by the QAC, the data handling portion of the 
audit consisted of a performance audit of cumulative and daily air con- 
centration data and a systems audit of sequential sampler methodologies 
and data handling procedures. 

A draft copy of the APIOS QA Manual (CSC, 1984b) has been sub- 
mitted to the APIOS management for review. As described in the Phase I 
audit (CSC, 1984a), this manual brings together all the various quality 
assurance procedures currently being carried out (e.g. co-located sam- 
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pling, calibration, blind samples, etc.) as well as describing QA acti- 
vities which have not yet been incorporated into the operating system 
(e.g. internal audits, corrective action, procurement control, etc.). 
This manual also decribes how these procedures fit together in the 
quality assurance system, so that estimates of data accuracy, precision, 
completeness, comparability and representativeness can be made. 

Finally, as stated in the Phase I audit, the auditors have 
noticed marked improvements in all areas of the programme, particularly 
in the areas of network documentation, data validation and the implemen- 
tation of additional routine QC procedures. With the completion of the 
QA manual, the full APIOS quality assurance programme can become opera- 
tional, and network data evaluated with respect to the network QA objec- 
tives and the standard QA elements of accuracy, precision, representa- 
tiveness and comparability. 



G )ix ( )r<,l bi. iLiiliiK C< irj)i xvilion 



2.1 - 



2. INTRODUCTION 

Concord Scientific Corporation (CSC) has undertaken on behalf 
of the Ontario Ministry of the Environment (MOE) two audits of its 
Acidic Precipitation in Ontario Study (APIOS) monitoring network (labo- 
ratory activities were not included in the audit since these aspects 
were extensively addressed in a previous audit, (CSC, 1982)). 

The Phase I audit was carried out during the fall of 1983 
through to the winter of 1984 and the Phase II audit was carried out 
during the summer of 1984. The findings of the Phase I audit were 
presented in the Phase I Report (CSC, 1984a) to the APIOS QA 
Coordinator. 

This report contains the findings of the Phase II audit inclu- 
ding recommendations for changes to the APIOS programme. Volume I of 
this report contains four main chapters as did the Phase I Report. 

The first chapter summarizes the audit results and points out 
major strengths and weaknesses of the programme. The third chapter is 
dedicated to performance related audit findings. These findings are a 
quantitative and qualitative measure of how well the APIOS procedures 
are being executed. The fourth chapter reports on system related audit 
findings, and gives details on modifications recommended to aspects of 
the programme which will assist in attainment of both network and 
quality assurance objectives (Bardswick, 1983b) and provide greater 
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efficiency in operations. The last chapter collates the recommendations 
described throughout the report in a summary form, as well as reviews 
the actions on the recommendations from the Phase I audit. 

Volume II of the Phase II Audit Report contains the completed 
Phase II site audit questionnaires prepared by CSC's auditors, including 
photographs of the audited sites. 
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3. PERFORMANCE RELATED AUDIT 

The purpose of a performance audit is to quantitatively evalu- 
ate the quality of the data produced by the total measurement system, 
e.g. sample collection, sample analysis and data processing (U.S. EPA 
1977). 

The performance audit carried out in Phase II is broken down 
Into three components, site audits, technologist audits, and data hand- 
ling and validation audits. For each of the components the network's 
performance with respect to documented APIOS procedures was described in 
quantitative and qualitative terms. The findings are discussed below 
and the recommendations from this chapter are summarized in Chapter 5. 

3.1 Status of Monitoring Sites 

Site evaluations were carried out at thirteen APIOS monitoring 
stations. The list of sites inspected is presented in Table 3.1. 

The site visits included audits and evaluations of the site, 
operator sample handling procedures, plus configuration and calibration 
checks on instrumentation. 

The questionnaires and audit check lists used during the site 
visits are identical to those used during the Phase I Audit, and are 
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TABLE 3,1 



Sites Visited During the Phase 11 Site Audits 



Site 



^ 



JXEl 



Merlin 

Port Stanley 

Pickle Lake 

Forbes Township 

Quetico Centre 

Ear Falls 

Experimental Lakes Area 

Dorset 

Whitney 
Coldwater 
Gowganda 
Winisk 
Whitman Creek 



Cumulative Wet 

Cumulative Wet % Dry 

Cumulative Wet & Dry 

Event Wet 

Cumulative Wet and Event Wet 

Cumulative Wet & Dry 

Cumulative Wet 

Cumulative Wet 4 Dry, Event Wet 

& Dry 

Cumulative Wet 

Cumulative Wet 

Cumulative Wet & Dry 

Cumulative Wet 

Event Wet 
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presented in Appendix I. The completed questionnaires, site and aerial 
photographs are presented under a separate cover in Volume II of this 
report. Site audit summaries are presented in Appendix II. These 
summaries include site evaluations, instrument evaluations, operator 
evaluations and corrective action required at each site. 

Table 3.2 summarizes the audit findings for each site. This 
table is broken down into two components, a physical siting related 
component and an operations related component. The guidelines utilized 
for the completion of this table are presented in Table 3.3. Each 
component was assigned a rating, and in order for these ratings to be 
comparable with the subjective evaluation performed by regional technol- 
ogists, the auditors used the APIOS Ranking Criteria (Table 3.4) as a 
guideline. A summary of the siting and operations ratings for all sites 
audited is presented in Table 3.5. 

As was described in Phase I, a marginal or poor rating 
requires that the QA Coordinator review the site and make appropriate 
changes to improve the rating or relocate the site. A marginal rating 
is an alert that potential problems exist and a poor rating indicates 
that changes should be made immediately. 

Sites that have been classified as poor are discussed in more 
detail below, and where applicable, site changes are recommended. The 
importance of the deviations from the siting criteria should be 
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TABLE 3.2 
Smwiary - "hase II Site Audit findings 





Ear Falls 


E.L.A. 


Ouetico Centre 


I SITING 








a) Windbreak 


180* (mixed) 


40* (burned Coniferous) 


330* (nrixed) 


bl Ground Cover 


Poor (clay t enposed 
earthl 


Good (bedrock, iross) 


Good 


c) Obstructions 


Yes (trees, gasoline 
storage tank) 


Met Tower H>10ra 14(n E 


None 


dl Rep. of Topography 


res 


res 


res 


e) Rojds/Tr«nsport«t1on 


Access road 4 parking 
lot (gravel) 


None 


Access rd. 30m M (paved) 


f) Agriculture 


None 


None 


None 


g) Towns/Cities 


Ear Falls (<1 )(m) 


None 


None 


h) Pollution Sources 


Open pit Iron ore 
mine 25km N 


None 


Hone 


i) Power Failures 


3 per year 


1-2 per year 


1-Z per year 


jl Deviations 


Poor ground cover, 
obstructions, 
projiinity to road, 
construction, open 
pit Bilne 


Tower - Influence is 
minimal 


Proximity to road 


k1 Advantages 


Site is accessible 
and secure 


Remote and represent- 
ative 


Remote and represent- 
ative 


1) Rating 


Poor - many deviations 


Excellent 


Excellent 


II OPERATIONS 








al Condition of Instr. 

- Present 

- Past 


Chain needs tighten- 
ing, limi t snitch 
needs to be replaced 
i adjusted 


Wotor humming otherwise 
good 

(^ood 


Good 
Sood 


b) Understanding 

- Has manual 

- Bead manual 

- Understands Protocol 


N/A 
OPERATOR 


Sections pertaining to 

him 

Yes 

Yes 


Yes 

Ves 
Yes 


c) Follows Protocol 

- Instr. Checks 

- Sample Handl ing 

- Cleaning 

- Storage [sample) 

- Std. Gauge Protocol 


WAS HOT 
PRESENT 

FOR 

THE 

AUDIT 


Daily 

Good - follows protocol 

good 

Follows protocol 


Instrument checked 
dally - well maintained. 
Operator needs to be 
retrained in sample 
handling - possible 
source of contamination, 
std. gauge taken 
properly 


dl Attitude 




Good 


Good attitude - but 
feels too much paper 
work 


el Reaction to Problems 




Good 


Good, telephones 
technologist 


f) Subaisslon Forms 




Properly filled out 


Properly filled out 


g) Retraining Needed 




None 


Yes - needs to be 
retrained on sample 
handl ing 


ti) Deviations 
i ) Advantage 




None 

Conscientious 
Interested in network 
would like more info on 
APIOS and complete 
operations manual 


Leans over pal1. same 
gloves used to pat 
down bag as to install 

- checks sampler daily 

- conscientious 


J) Rating 




Excellent 


Good (retraining) 
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TABLE 3.2 (cont'd) 
Sumpiary - Ptuse II Site ft-udit Findings 



I SITING 

a) Windbreat 

b) firound Caver 

c) Obstructions 

d) Hep. of Topograpftjf 
t) Roads/Ti-ansportation 

f) Agriculture 

g) Towns/Cities 

h) Pollution Sources 

II Power Failures 
j) Deviations 



*) Advantages 



11 Rating 



Dorset 



360* (mixed) 

Compound - good; 
surrounding fair 



Hone 

Mi 

Driveway - dirt 50m 
ENE 

MOM 

None 
None 

Z per year 

Proximity to driveway 
Idlrtl 

Trailers between 
parting lot and site 
to reduce influence 
of Kindblown contami- 
nation 

Parking lot should be 
paved 

Meets ill siting 
criteria 

Excellent 



Whitney 



360* (mixed) 

Marginal I some exposed 
earth weeds need 
cutting) 



Hone 






None 


Yes 






Yes 


None 






None 


Xone 






Yes-corn/hay fSOOin-lkn 
ENE) 


None 






Oril lia Ukm pop. 
25 ,000 


Sawmill 

Marsh 


35 km SE 
(downwif 
35ni NE 


d) 


Open pit limestone 
quarry I3-15kiii N£ 


3-3 per 


year 




None 



Exposed earth, proximity 
to marsh (35mi and 
sawmi 11 



Remote, representative, 
accessiSility of power 
and operator 

Marginal, (good if 
ground cover Improved) 



Coldwater 



360° (deciduous) 
Sood, cut grass 



Proximity to limestor 

?uarry and plowed 
ie1ds 



Meets most siting 
criteria 



Good (*evalyate 
influence of limestone 
quarry) 



n OPERATIONS 



a) 


Condition of Instr. 






- Present 


Primary good; 
secondary need 
maintenance 




- Past 




b) 


Understanding 






- Has manual 


Yes 




- Read manual 


Yes 




- Understands frotocol 


Yes 


c) 


Follows Protocol 






- Instr. checits 


Yes 




- Sample han<Jl ing 


Does not Change 
gloves to pat down 
bag 




- Cleaning 


Yes 




- Storage tsample) 






- Std. Gauge Protocol 


Follows protocol 


d) 


Attitude 


Very good 



e) Reaction to Problems 
fl Subaiission Forms 
gl Retraining Needed 

h) Deviations 
1) Advantages 

i) Rating 



Filled out correctly 

No - just needs 
refresher 

Pats down bag 
without changing 
gloves 

AP105 Regional Tecti. 
- follows almost all 

protocol ; good 

attitude 

Good 



Good; clutch needs 
adjustment 



NM 

OPERATOR 



UAS 
NOT 
PRESENT 
FOR 
THE 
AUDIT 



Good 



Good 



Yes 
Yes 

Yes 



CrnM < 



Zx per week 

Does not follow protocol 



Yes - no gloves 

Porcn 

Good 

Excellent 

Good 

Filled out correctly 

Yes 



Folds bag in before it 
is tied 



Does not label bag; 
does not always use 
new gloves 



i[)tenj1iMl! (needs retrain- 
ing) 
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TABLE 3.2 (cont'd) 
Sunmary - Phase II S<te AutUt Findings 



I SITING 

4) Windbreak 

bl Ground Cover 

c) Obstructions 

d) Rep. of Topography 

e) ao»ds/Tr«Bsportatior 

f) Agriculture 

9) Towrs/Cities 

h) Pollution Sources 

i I Power Failures 

j) Deviations 

kl Advantages 

11 Rating 



Hhitman Creek 



270* (deciduous) 



Good (grass needs to 
be cut) 

None 

res 

res, Count;^ Rd. #14 
ZSir S 

Hone 



Hont 

S«wni11l lOlim E 



Many - battery 
operated (solar cell 
source) 

ProJiiiflity to road 
and sawmill 
unreliable power 
source 

Representative, 
accessibility, good 
ground cover, no 
obstructions 

Marginal (electrical 

power) 



Port Stanley 



180' (deciduous) none 
in prevai ling wind 
direction 

Good 



None 

Access rd. (20b W gravel 
very light) 

res (but representative 

300ni) 



None 



Treatment plant, oil 
heated 



Merlin 



3 per month 



Cultured fields within 

1 km 

Proxlitity to roads 



Representative, no 
obstructions good ground 
cover 



Good 



180° (scattered houses, 
barns 1 poor 



Good 



Hone 

Yes 

County rd. 50m S; gravel 
access rd. IS m E 

Tes (hut representative 
SOOm, 1km) 



Merlin <1 km 

Chatham ISksn N (10.000) 

None 



None, back up generator 



Proximity to vehicular, 
agricultural and human 
activity 



Representative, good 
ground cover, no 
obstruction or 
pollution sources 

Good 



11 OPERATIONS 

a) Condition of Instr. 

- Present 

- Past 



b) Understanding 
' Has manual 

- Read manual 

- Understands Protocol 

c) Follows Protocol 

- Instr. Checks 

- Sample Handl ing 

- Cleaning 

- Storage (saraplel 

- Std. Gauge Protocol 

dt Attitude 

e) Reaction to Problems 

f) Submission Forms 
9) Retraining Needed 
h) Deviations 

1) Advantages 

j) Rating 



Good sensor sensi- 
tivity needs adjust- 
ment 

Poor unreliable 
during winter aonths 



Yes 

Yes 
Yes 



res 

Follows protocol 

Refrigerator 
Follows protocol 

Excellent 

Good - informs tech, 
of any problems 

Completed thoroughly 

No 

None 



Conscientious, good 
attitude 

Excellent 



Good sensor sensitivity 
needs adjustment 

Good - but some problems 
with circuit breaker 
tripping 



Yes 
res 
Yes 



Oally 

Uses only I pr.of gloves 
Cleans hood gasket 
Cabinet In laboratory 
Follows protocol 

Excellent 

Good - Informs tech. of 
any problems 

Completed thoroughly 

Yes 

Does not wear new gloves 
to touch insid* of bag 

Good attitude 



Good 



Good sensor sensitivity 
neeas adjustment 

Good 



res 
Yes 
res 



1 per week 
Follows protocol 
res 

SoK in pump house 
Follows protocol 

Excellent 

Good 

Completed thoroughly 

No 

None 

Good attitude follows 
all protocol 

Excellent 
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TABLE 3.2 (cont'd! 
iumtry - Phast II Site AudU Findings 



I SITING 

a) Windbreak 

b1 Ground Cover 

e) Obitructlons 

dl Rep. of Topography 

el Roads/Transportation 

f) Agriculture 

g) Towns/Cities 

hi Pollution Sources 

1) Power Failures 

Jl Deviations 



k) Advantages 
1) Rating 



Forbes T«ip 



360* (mixti) 

Sood 

None 

Tes 

None 

None 

None 

Power plant (35 lu«> 
(downwind I 

3 per year 

Proniwlty to power 

plant 

Remote, representative 
Good 



Pickle Lake 



Winlsk 



JOO* (Coniferous) 

Good 

Trees (E, SE:) Tower (HE) 

Yes 

Hwy. 646. *6m I {vtry 
light) 

Kone 

pickle Lake (<1 km) 

Wood burning source of 
heat 

None 

Obstructions, wood 
burning source of heat 

Remote, representative 

5ood - trees should be 
removed 



270* (Coniferous) 

Good 

None 

Yes 

Snowmobile road Im, 
Airport Hanger (*00ra) 

None 

None 

Diesel generator 
m tirs/dayl 

None (back-up generator) 

Pro»1mity to snowmobile 
trail, airport hanger i 
runway, diesel generator 

Heiaote, representaive 

Marginal 



II OPERATIONS 

a) Condition of Instr. 
- Present 



- Past 

b) Understanding 

- Has manual 

- Read manual 

- Understands Protocol 

c) Follows Protocol 

- Instr. Checks 

- Sample Handling 

- Cleaning 

- Storage (sample) 

- Std Gauge Protocol 



d) Attitude 

e) Reaction to Problems 

fl Submission Forms 

g) Retraining Needed 

h) Deviations 



1] Advantages 

j) Rating 



Good 



No problems 



Yes (event section) 

Yes 

res 



Yes 

Follows protocol 

Ueekly 

Refrigerator 
Follows protocol 



Excellent 

Good - frequent 
contact with 
technician 

Completed thoroughly 

No 

None 



Conscientious i 
follows protocol 

Excellent 



Clutch too tight >35 lbs Not operating at time of 
sensor grid blackened audit 



No Baintenance performed 



Yes (cumulative section) 

Yes 

Yes 



Yes - dally 

Follows protocol 

Cleans gasket but no 

gloves 

Refrig, until shipped 

Follows protocol 



Poor - temporary oper. 

Good 

Completed thoroughly 

Ho 

None 

Follows protocol 

Good 



Yes 

No 
NO 



None 

Does not follow protocol 

None 

Kept In repair shop 
Std. gauge empty at 
tine of audit 

Very poor 

Does not call when 
Instruments down 



Not filled out properly 

Yes 

numerous - too many to 
list 

Hone 
Poor 
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TABLE 3.2 (cont'dl 
Sunrnary - Phase II Site Audit Findings 





Gowganda 






I SITING 








a) Windbreak 


165' (mixed) 






b) Ground Cover 


Good (weeds to be cut) 






c) Obstructions 


Tower (H-lZm 20in SE) 






d) Rep. of Topography 


Yes 






e) Roads/Trdnsportation 


Access Rd. gravel 25ni 
S i E 






f) Agriculture 


None 






g) Towns/Cities 


Gowganda < 1 km 






h) Pollution Sources 


None 






i) Power Failures 


Seldom 






j) Deviations 
k) Advantages 


Poor windbreak 

especially on Lake 

side 

vandalism 

Remote, representative 






1) Rating 


Good (improve 
windbreak on lake side 
cut grass 2x year) 






II OPERATIONS 








a) Condition of Instr. 

- Present 

- Past 


Good 
Good 






bJ Understanding 

- Has manual 

- Read manual 

- Understands Protocol 


Yes 

Yes 
Yes 






c) Follows Protocol 

- Instr. Checks 

- Sample Handling 

- Cleaning 


Yes 

Follows protocol 
Does not clean sensor 






- Storage (sample) 

- Std Gauge Protocol 


grids 

In insulated box 
Follows protocol 






d) Attitude 


Very good 






el Reaction to Problems 








f) Submission Forms 


Filled out completely 






g) Retraining Needed 


Operator unsure of 
sampling procedures 
should be retrained 






h) Deviations 








1] Advantages 








j) Rating 


Excellent 
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TfflLE 3.3 
SufdeHnes for tht Cowpletlon of 'Suwwry - Ph»se U S1t« Audit Findings" 



Coaconent 



I SITING 

a) ulndbreak 

b) Sround Cover 

c) Obstructions 

d) Rep. of Topogripfiy 

e) Ro«ds/Transport»t1on 

f) Agriculture 

g) Tonns/Cltles 
h) Pollution Sources 

1) Power Failures 

i) Deviations 

k) Advantages 

11 Rating 



Suldellnes 



At how imnji degrees of the conpass Is there a windbreak and describe type? 

Good ground cover Is cut grass or boss; poor ground cover Is exposed 
soil. 

No object should be within Z.S t1»es object height to sampler. 

The topography and vegetative grognd cover should be representative of 
surrounding area (SO km] . 

No major transportation line (road/rallwy) within one kllopieter of site 
as it may be a contamination source. 

No agriculture which luy provide a contamination source within one 
klloaetre of the site. 

No urbanized areas close to the sampling site (distance depends on 
size of town/cltyl . 

No minor pollution sources within 1 km (e.g. sewage lagoons, salt and 
sand piles, gravel pits). No major pollution sources witfiin SO km 
(power plants, smelters, open pit mines, pulp and paper plants, etc.). 

How many power failures are there per year? 

Nhat are the deviations of the site from the APIOS siting criteria? 

Nhat are the advantages of the site? 

Objective evaluation based on finding fro« Item (a) to (k) above. 



I] OPERATIOKS 

a) Conditions of Instr. 

- Present 

- Past 

b) Understanding 

- Has Manual 

• Read Manual 

- Understands 
Protocol 

c) Follows Protocol 

- Instr. Checks 

- Sample Handling 

- Cleaning 

- Storage (sample) 

- Std.I^auge Protocol 

d) Attitude 



e) Reaction to Problens 

f) Subnisslon Forms 

g) Retraining Needed 

h) Deviations 

i) Advantage 

J) Rating 



findings of tests and observations carried out during the site audit 
to determine the past and present performance of the instruments. 
The information Is based on calibration and configuration cheeks, 
operator comnents and auditor's observations. 

Understanding by the operator of procedures. Has he read the APIOS 
manual, does he understand It, does he understand the sampling 
protocol? 



Does the operator check the performance of the Instruments, handle 
the sample as per protocol, clean the required cainponents properly 
(knife edge, gasket), store sample 1n a location that would prevent 
sample degradation and follow only updated saaipliog protocols? 



Operator's attitude towards the procedures that he/she nwst follow and 
towards the APIOS program in general. 

Does the operator get in touch with the regional technologist promptly If 
a problem Is noted? 

Are the submission forms filled out properly and coi(pletely? 

Is the operator following aost current procedures? Is retraining 
required? 

What procedures performed by the operator deviate from current APIOS 
operator sampling protocol? 

Site operations advantages (e.g. attitude, protocol followed). 

Objective evaluation based on the findings fro* (a) to (1) above. 
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TABLE 3.4 
APIOS Ranking Criteria 



Rating 


Site 


Operator 


Excellent 


An excellent site should meet all siting 
criteria. It is easily accessible 
year-round and has no record of any 
vandal 1 sin. 


An excellent operator is one who 
follows APIOS handling procedures at 
all times. Samples are collected as 
scheduled, routine instrument checks 
are made, and detailed log books are 
maintained. The operator maintains 
an interest in the program and 
regards his/her role as an important 
function. The excellent operator is 
one who goes out of his way to ensure 
the collection of quality samples. 
These operators frequently 
participate in special studies. 

-„- 


Good 


These sites meet the majority of siting 
criteria. Any deviations from siting 
criteria do not significantly affect the 
integrity of samples collected. Examples 
of this category might include sites that 
do not have windbreaks on all compass 
points; sites located near agricultural 
areas; sites that have accessibility 
problems during certain times; or sites 
that may have objects <2.5 times the 
height away from the sampler {but not <1 
time the height away). 


A good operator follows APIOS proce- 
dures. Samples are collected, rou- 
tine instrument checks are made, and 
i log book is maintained. The opera- 
tor may not be sincerely interested 
in the program but procedures are 
followed as required. 


Marginal 


These sites need improvement in order to 
be considered a good site. Examples 
might be sites with marginal ground 
cover, or sites with object(s) near the 
sampler (1 time the height) that may 
affect the sampler occasionally. Data 
from these sites should be closely 
scrutinized. 


Marginal operators do not always 
follow standardized procedures. They 
occasionally forget to change samples 
and keep sketchy log books, if any. 
The low interest of these operators, 
combined with their performance, do 
not allow the collection of quality 
data on a regular basis. 


Poor 


These sites should either be removed or 
significantly improved. They do not meet 
important siting criteria and samples are 
routinely contaminated. Data collected 
is unreliable. 


Operators in this category do not 
follow procedures and maintain no 
interest in the program. If 
performance cannot be upgraded, then 
an alternative operator should be 
immediately located. If no other 
operator can be found, consideration 
should be given to removing the 
site. 
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Siting 



Operations 



Excellent 



Good 

Marginal 

Poor 

Unrated 
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TABLE 3.5 
Summary of Site Audit Ratings 



Excellent 



E.L.A. 



Dorset^ 
Quetico 
Centre^ 



Good 



Gowganda 
Merlin 
Forbes Twp 



Port Stanley 
Pickle Lake'' 



Col dwater"* 



Marginal 



Whitman Crk. 



Poor 



Winisk^ 



Whitney- 



Ear 

Falls^ 



1. Unreliable power source, numerous events missed. 

2. To reduce influence of windblown contamination, parking lot should 
be paved. 

3. Good if ground cover improved. 

4. Evaluate influence of limestone quarry; retrain operators 
immediately. 

5. Many deviations, site to be reviewed by QA Coordinator. 

6. Operator needs upgrading. 

7. Obstructions (trees) should be removed. New operator to be 
trained. 

8. Poor operator; should be replaced. 
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evaluated. Action to correct the faults and/or qualify the data collec- 
ted at these sites should be taken. 

1. Ear Falls 

The cumulative monitoring site at Ear Fa>ls was rated as 
"poor" due to numerous deviations from the APIOS siting criteria. These 
deviations Include: 

poor ground cover (gravel & dirt) 

on-site obstructions Include two coniferous trees 10m NE and a 

gasoline holding tank 8m NE 

proximity to a small industrial park {3 km W) 

frequency of vehicular traffic near the gasoline holding tank 

location of site (adjacent to a utility garage and storage 

area) 

It is strongly recommended that the data collected to date be 
carefully evaluated and this site be relocated in order to collect more 
representative samples. 
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Winisk 



The operations component of the Winisk site was rated as poor. 
The Sangamo collector was in poor condition at the time of the audit. 
The GFIC (ground fault interruptor circuit) had tripped, the fuse was 
blown, one of the sensor grids was broken, the clutch was set too high 
and the old knife edge collar (with a gap) had not been replaced. 

Historical data from this site should be carefully examined. 
The auditor suspects that bulk samples were being collected at this site 
during the last few months (if not longer). Samples are often collected 
weeks and even months late. The operator lacks interest and has a poor 
attitude towards the APIOS programme. As well, he does not follow any 
of the documented sample handling procedures. 

Deviations from the siting criteria include the proximity of 
the site to the snowmobile road and the airport. However, given the 
remoteness and operator accessibility, it is a reasonable compromise. 
The site could be improved by limiting the snowmobile traffic. 

Operator and communication problems at this site are extreme 
and lead to an overall unacceptable rating. The auditor believes that 
it would be very difficult to train this operator due to attitude. 
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communication and other problems. It is recommended that a new operator 
be found and thoroughly trained, otherwise the site should be removed. 

Some general observations made during the audit are discussed 
below: 

1. The majority of the operators audited have shown marked 
improvements in the performance of the sample handling proce- 
dures outlined in the APIOS Technical and Operating Manual 
{Bardswick, 1983). Most technologists have either retrained, 
upgraded and/or reviewed the current operating procedures with 
their operators. At some sites, the "procedural short list" 
from the manual has been posted near the instruments. This 
allows the operator to quickly check that he is following the 
procedure correctly. Posting procedural short lists should be 
considered at all sites. 

2. The following inconsistencies were noted in sampler configura- 
tion. 

a) The southwestern and central regions had no aluminum 
foil wrapped around the collection vessels to reduce 
sample loss due to evaporation. 
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b) Aerochem Metrics dry buckets were still in place in the 
northwestern region. This was the only region utilizing 
these buckets. 

Uniformity of instrument configuration throughout the network 
should be instituted. Inconsistencies in configuration from 
region to region should be assessed by the QA Coordinator for 
their effect on sample quality. 

3. Table 3.6 summarizes the instrument checks performed during 
the instrument audits. In general, most instruments were 
operating within the specifications set out by APIOS, How- 
ever, the co-located Sangamo Type A precipitation collector in 
Dorset requires immediate attention. 

4. To improve the seal between the collector hood and the bucket 
of the cumulative precipitation collectors, new gaskets have 
been attached to the underside of the Sangamo/M.I .C. hoods at 
all cumulative sites (except Dorset because of an on-going 
precision study). These new gaskets provide a tight seal and 
prevent both sample loss due to evaporation and contaminants 
from entering the sample collection containers. 

5. Revised site documentation for the central region has not yet 
been completed. This should be carried out as soon as 
possible. 
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TABLE 3.6 
Iflstrunent Perforwance Audit Su—ary 



i 



Regton: Central 





HIC 


Orifice 


Collec- 
tor 




Hood 


Instrument 
Delay Time 


Instrument 


Sensor 

Grid 


Clutch 
Adjust- 


Limit Snitch 






Storage Gauge 




Orifice 




Site 


Serial f 


Height 


Area 


Level 


Hovement 


(sec) 


Sensitivity 


Condition 


ment 


Condition 


Gasket Seal 


Coimiwnt 


Height 


Siie 


Level 


Whitney 


HIC 
Type A 


2 ■ 


flound 


let 


-Hovement 

smooth 
-Hood does 

drof) on 

knife edge 


100 


ZOO KVJ 


Good 


>3S lbs. 


-Needs 

adjustment 
-Too much 

pressure on 

support 
bracket 


Good 

-Hew gasket 
Installed 


Tech Installed 
mesh screen on 
storage gauge 
to keep debris 
1 birds out 


1.32 • 


Round 
28 cm 


Yes 


Dorset 


Prlaary 

HIC 
Type A 


1.96 HI 


Round 
20 cm 


fes 


Good 


79 


200 Kn 


Good 


>35 lbs. 


Good 


Good 

-Old gasket 
used for 
special 
studies 




1.26 m 


Out of 
round 


No 




Secondary 


1.9S ■ 


Round 


res 


-Not smooth 


86 


354 icn 


Good 


>35 lbs. 


-Needs 


Good 


Sangamo 


t.24 m 


Out of 


Ho 




HIC 




20 cm 




-Drops 










adjustment 




• Off switch 




round 






Type A 








suddenly 










-Improper 
contact 




not working 

• Motor runs 
on after 
Instrument 
Is turned 
off 








Cold- 


HIC 


2 ■ 


Round 


Tes 


Kood 


86 


260 K)l 


Good 


14 lbs. 


Good 


Good 




92 cm 


Out uf 


res 


water 


Type A 




20 cm 
















-Hew gasket 






round 





r 
I 



TABLE 3.6 (cont'd) 
Instruwent Perfonaance Audit Suiiwary 



Region: Horthwest/Northeist 



n 






site 


NIC 
Serial f 


Orifice 

Height 


Collec- 
tor 
Area 


Level 


Hood 
MoKement 


Instruaient 

Delay Tine 

(sec) 


Instrument 
Sensitivity 


Sensor 

Grid 

Condition 


Clutch 

Adjust- 
nent 


Linit Switch 
Condition 


Gasket Seal 


Comnent 


Storage Gauge 


Height 


Orifice 
Stie Level 


t.L.A. 


HlC 
Type A 


2 ■ 


Round 
20 en 


No 


Good 


SO 


200 Kn 


Good 


>30 lbs. 


Good 


Good 
-New gasket 


*Fol) wrapped 
around long 
tube 

•Motor hunmlng 


l.«0 ■ 


Round 
28 CB 


res 


ear 
FtDs 


HlC 
Type A 


2. 


Round 
20 ca 


res 


-Hood drops 
on both wet 
I dry side 


80 


N/A 


Good 


20 lbs. 


-Needs adjust- 
■ent 

-Too such pres- 
sure on limit 
swIUh bracket 


Good 
-Hew gasket 


•Foil wrapped 
around long 
tube 


l.SO n 


Round 


res 


Quettco 
Centre 


MIC 
Type A 


2 m 


Round 
20 en 


res 


Good 


87 


200 KQ 


-Hot to 

touch 
-One grid 
blackened 


20 lbs. 


Good 


Good 
-Hew Gasket 


•Foil wrapped 
around long 
tube 








Ptckle 

Lake 


HIC 
Type A 


; ■ 


Round 
20 cm 


Yes 


Good 


110 


N/A 


-One grid 
blackened 


>35 lbs. 


Good 


Good 
-New gasket 


•Foil wrapped 
around long 
tube 


1.30 ■ 


Round 

30 CM 


Yes 


WInUk 


HIC 

Type A 


2 ■ 


Round 

19 CM 


No 


Good 


30 


400 Kn 


-One grid 
broken 


>3S lbs. 


Good 


Good 

-Old knife 
edge 
collar 


•GflC tripped 
1 fuse blown 

-One socket 
exposed to 
nolsture 








Gowgdnd^ 


MIC 
Type A 


2 in 


Round 
19.5 cm 


Yes 


Good 


3S 


23 Ka 


Good 

very Mam 


>3b lbs. 


Good 


Good 
-New gasket 




1.30 ■ 


Round 
29.Sc« 


Yes 



I 

I 



TABLt 3.6 (cont'd) 
Instrument Performance Audit SuMwry 



Region: SouthMCtt 





NIC 


Orifice 


Collec- 
tor 




Hood 


Instrument 
Delay Time 


Instrument 


Sensor 
Grid 


Clutch 
Adjust- 


Limit SHltch 






Storage Gauge 




Orifice 




Site 


Serial 1 


Height 


Area 


Level 


Hovenent 


(sec) 


Sensitivity 


Condition 


ment 


Condition 


Gasket Seal 


Coment 


Height 


Sfie 

1 


Level 


Merlin 


HIC 
Type k 


1.99 ■ 


Round 

20 CM 


Yes 


Good 


106 


100 11 


Good 


IS lbs. 


Good 


Good 
-New gasket 




1.4a m 


Round 
20 cm 


»es 


Port 


Prlatry 


1.93 ■ 


Round 


res 


Good 


95 


40 n 


Good 


16 lbs. 


Good 


Good 




1.61 ■ 


Hot 


Tes 


Stanley 


HIC 

Type A 




20 CM 
















-New gasket 






round 
29.2 cm 
2B.8 cm 






Secondary 


1.96 ■ 


Round 


Yes 


-Not SKMlth 


90 


100 


Good 


16 lbs. 


Good 


Good 


•GfIC trips 


1.73 > 


Round 


Tes 




MIC 




20 CM 




-Jerks 












-New gasket 


frequently 




28.6 cm 






Type A 

1 






















up to 3 X 

per month 









I 

03 



n 

X 

i. 



r; 



TABLE 3.6 (cont'd) 
Instrument Performance Audit Summary 





Aerochem Metric 


Rain Gauge 


Site 


Sampler 
Orifice 


Level 


Comments 


Gasket 
Seal 


Gasket 
Condition 


Sensor 
Condition 


Orifice 
Height 


Level 


Clean 


Forbes Twp. 
Primary 


Round 
28.5 cm 


Yes 


- On/off switch 
installed 

- Dry bucket still in 
place 


Good 


Good 


Good 


43 cm 


Yes 


Yes 


Forbes Twp. 
Secondary 


Round 
28.5 cm 


Yes 


- On/off switch 
installed 

- Dry bucket still in 
place 


Good 


Good 


Good 


44 cm 


Yes 


Yes 


Whitman 
Creek 


Round 
28.5 cm 


Yes 


- Power problems 

- Many events missed 


Good 


Good 


Good 


76.5 cm 


Yes 


Yes 


Quetico 
Centre 


Round 
28.5 cm 


Yes 


- Dry bucket in place 

- Water dripping from 
sensor head 


Poor 


Moisture 
on inside 
of gasket 


Good 


42.5 cm 


Yes 


Yes 



I 

I— • 
I 



It' 

3 
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3.2 Technologist Performance 

The APIOS technologists carry a major portion of the responsi- 
bility for ensuring that the network operating procedures are correctly 
carried out. They must ensure that the instruments are working properly 
and that all relevant data are recorded and reported. 

During the Phase II Audit, two of the regional technologists 
were visited, one in the northwestern region and one in the central 
region. The findings of the technologists' audit indicate that the 
regional technologists were following operating and technical procedures 
as described in the APIOS Technical and Operating Manual (Bardswick, 
1983). 

The technologists maintain close communication with the opera- 
tors in their region through frequent telephone contact and site visits. 
Event sites are visited once a week, and cumulative sites at least once 
a month in the central region, and once every two months in the 
northwest region. The technologists, where possible, meet with the 
operators during these visits and discuss sampling and instrument prob- 
lems. They also provide their operators with literature on "Acid Rain", 
such as MOE publications, articles from periodicals and newspapers, and 
will answer any questions the operators may have. The operators find 
these publications and discussions informative and technologists should 
be encouraged 
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to participate in this type of information exchange. It Is recommended 
that the technologists continue to maintain close conmunication with 
operators, encourage operators to contact them for any reason, and try 
to co-ordinate their site visits so that they meet with the operators as 
often as possible. 

Technologists have either reviewed operating procedures or 
retrained their operators within the last four months. Although 
operators were, for the most part, following standard operating proce- 
dures, it is recommended that the technologist carry out a procedures 
audit at least twice a year to ensure that operators are following the 
current protocols. Technologists should ensure that the operators are 
not contaminating the sample while performing these procedures. 

In the northwestern region where sites are widespread, and the 
visit frequency is low (every few months) due to travel restraints, the 
regional technologist requires duplicates of operator sheets, monthly 
site summaries (see Figure 3.1) and log books. One copy remains at the 
site and the second copy is sent to the technologist. The purpose of 
this is two-fold: 

a) It allows the technologist to get an overall view of what is 
happening in the region, detect any problems, and schedule 
site visits. 
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FIGURE 3.1 
Operator Honthly Site Sunmary Sheet 



? 

R 



r, 



Collection Period 
Operators Cumulative Log Sheet From: To: Site Name: 


Instrument 


Date 


Time 


Operation 
Check 


Storage Gauge 


Volume 
(L) 


Time 
(Hrs) 


Rotameter 
(Ipm) 


Coirments-Observatlons 


Depth 


Tewp. 
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b) It provides a QC check to ensure that the operator sheets are 
filled out correctly and completely and that all relevant 
sample information is documented, e.g. identifying potential 
sources of sample contamination, instrument malfunctions, 
etc. 

It is recommmended that this system be adopted by the network where 
sites are infrequently visited by technologists due to time and travel 
restraints. 

The northwestern regional technologist has also developed 
Quality Assurance Activity Log Sheets (see Figures 3.2, 3.3) for the 
cumulative and event QA activities carried out in the region. This 
system enables the technologist to plan QA/QC activities necessary to 
meet the network requirements and at the same time monitor the 
effectiveness of the QA/QC system in his region. This system should 
also be considered for adoption by the network. 

The northwestern Regional Office provides the regional techno- 
logist with a student for two to four days a week. The majority of the 
student's time is spent on sample processing (precipitation samples and 
filters). Initially, the technologist should check all of the work 
carried out by the student (e.g. checks field submission sheets, filter 
loading and unloading procedures, etc.) to ensure that: a) the student 
has been trained adequately and b) the quality of the work is accept- 
able. 
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FIGURE 3.2 
QA Activity Log Sheet - Event Sampling 











Region 


Quality Assurance Activity Log - Event Sampling 






Year Technician 


Activity 


Sent out or 

Requested 


Done 




Blank 
Bags 


June 






Sept. 






Dec. 






Mar. 






Handling 

Filter 

Bags 


June 






Sept. 






Dec. 






Mar. 








Site 


Otter Quetico 
Forbes Island Centre 
6081 6111 6071 


Fernberg 
6051 


Operator Audit 


Field 
Blanks 
and 
Duplicates 


Mar. 






Apr. 






Hay 






June 






July 






Aug. 






Sept. 






Oct. 






Nov. 






Dec. 






Jan. 






Feb. 






Composites 


Feb. 






May 






Aug. 






Nov. 
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FIGURE 3.3 
QA Activity Log Sheet - Cumulative Sampling 



n: 

n 

i. 



Region 




Qualii 


ty Assurance Activity Loq - Cumulative SanpHng 
Year Technician 














Sent out or 
Requested 


ACTIVITY 


Blank 
Bags 


Hand! ing 
Filter 
Blanks 


Volume 

Measurement 

Checks 


Composites 


6011 


6121 


6031 


6041 


6051 


6071 


6091 


61U 


6061 


Jun. 




1^ 






















Sep. 


























Dec. 


























Mar. 


























Operator 

Auflit 
1M4 



























I 



Ln 



0' 
■5 
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The technologist performs scheduled maintenance, repairs, 
calibrations, QA/QC activities, data screening and validation, upgrading 
of sites and arranging operator and site visits. The technologist 
feels, however, that the "hands on" training he received on some 
instrumentation was inadequate (specifically, the sequential sampler and 
modified Aerochem Metric sampler). This technologist should be 
retrained on some instrumentation and the auditor suggests that this 
retraining be carried out in conjunction with the quarterly APIOS 
Technologist meeting. 

The APIOS Technical and Operating Manual (Bardswick, 1983) 
lists spare instrument parts that the technologists should have on hand 
to be able to quickly maintain their instruments. This list is based on 
maintenance requirements for five precipitation collectors: 

1 precipitation sampler 1 motor 

2 printed circuit boards 10 limit switches 
4 sensor grids 1 gasket 

2 capacitors 

The technologists were found not to comply with this 
guideline. The technologists either did not order sufficient supplies 
on their 6 month request forms, or when they did run out of a particular 
item they waited until the last moment to notify the Head Office 
Technologist. As a result, instrument downtime is longer than necessary 
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due to the length of time required for delivery of parts from Head 
Office; often two weeks are required to receive the necessary parts for 
repair of samplers. In order to maximize data completeness by reducing 
downtime, technologists should ensure that they have on hand at least 
the recommended supply of spare parts as a minimum at all times. 

The technologist in the central region is responsible for the 
operation and maintenance of a special studies site in Dorset. This 
involves operating and maintaining many co-located instruments such as 
sequential samplers, low volume air samplers, daily and cumulative 
precipitation samplers, gauges, continuous monitors, as well as conduc- 
ting various on-going special studies. 

The majority of his time is spent collecting, processing and 
documenting APIOS routine and special studies samples. With the addi- 
tional responsibility of installing, operating, maintaining and co- 
ordinating long and short-term special studies, the technologist Is 
often unable to perform regularly scheduled preventive maintenance 
procedures, and as a result some of the instruments in this region have 
not had preventive maintenance carried out. It is essential that all 
instruments be serviced and calibrated on a routine basis to ensure 
reliability and accuracy. The instruments in this region should be 
serviced and calibrated when required. If the technologist is unable to 
carry out this task then manpower should be made available to do so. 
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The technologists visited during the audit (Phase I and Phase 
ID have proven to be a real asset to the APIOS programme. Because of 
the unpredictable nature of field work, these technologists often make 
valuable subjective decisions regarding the operations of instruments, 
performance of operators, procedures for data handling and relevant data 
recording. They were found to be conscientious and concerned about 
their duties and generally their performances have ensured the genera- 
tion of good quality data. 

3.3 Performance Audit of Data Handling and Validation Procedures - 
Daily Air Concentration Data 

Data validation and screening systems are used to evaluate the 
raw data before including them in the final database. A number of 
screening techniques, checks and statistical routines are run on the 
data to flag unusual or suspect data. A careful scrutiny of these 
suspect data is performed before the decision to include these data in 
the database is made. Specific data validation procedures have been 
developed and documented for daily ambient air data and are described 
elsewhere (Kirk, 1984). Basically, the validation procedures utilize 
two approaches. The first approach is to apply gross limit checks to 
identify unlikely values and the second approach is to examine the 
sample structure by principal component analysis. 
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The performance audit checked whether the documented screening 
and validation checks were being properly carried out. Errors in the 
data handling system were detected with the use of non-routine manual 
screening techniques. 

The audited data set consisted of data collected at Fernberg 
and Longwoods during the months of February, March, June and July, 
1982. 

A description of the checks performed, along with the 
findings, are described below. 

1, Check - Field sheets were screened manually to ensure that 
they had been filled out completely and correctly by the regional tech- 
nologist. The field sheets contain information about the samples sub- 
mitted, such as the site name and number, sampling period, filter type, 
gas meter reading, sample volume, rotameter reading, sampling port 
number, sampling start and end dates, associated passive sample numbers 
and field comments about the sample. A blank air sampling field data 
sheet is shown in Figure 3.4. 

Findings - Fernberg 

• All field sheets for the months of February and March were 
missing associated passive sample numbers and sample type 
(e.g. active, passive or blank). 

G)n(tml Si imtitu CnifHjr.UKin 



- 3.30 - 

FIGURE 3.4 
API OS Air Sampling Field Data Sheet 
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• Filter lot numbers for Teflon, Nylon and Whatman 41 filters 
were missing for the period February 1-10, 1982 and the filter 
lot number for Teflon filters was recorded as a "question 
mark" for the period June 22 - July 6, 1982. 

• Associated passive sample numbers were recorded incorrectly on 
two occasions: 

Reported Associated Actual Associated 
Sample Number Passive Number(s) Passive Number(s) 

95441 95432/95448 95440/95448 

95443 95442 95448 



« 



All field sheets- for the study period were missing initial and 
final rotameter readings and operation check. 

Some switched filter dates for "set on tower" are recorded 
incorrectly. The switched filters are left on the tower for a 
two week period, the first week in port #8 and the second week 
in port §1. On some field sheets the date "set on the tower" 
was recorded as the first day of the second week (however, 
this date is not entered into the database). 



• All sample dates, times, station identifiers and sampling port 
numbers were recorded correctly. 
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Longwoods 

• Sampling end date for sample number 15347 was recorded in- 
correctly as 15/2/82. This was a two day sample beginning on 
14/2/82 and ending on 16/2/82 (sample volume = 52680 1). 

« All station identifiers, sampling port numbers and associated 
passive numbers had been recorded correctly. 

9 Some data recorded as "date set on tower" were incorrect for 
switched filters (see Fernberg). 

Manual screening of field data sheets prior to submission 
would have prevented omissions and errors. It is recommended that the 
technologists visually check the field sheets for missing information. 
Incorrect dates. Incorrect times and illogical coding prior to submis- 
sion. These manual screening techniques are described in the APIOS 
Technical and Operating Manual along with procedures to be followed when 
filling out these data sheets. When an omission or error is found, a 
revised or corrected field sheet should be submitted. 
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2. Check - Field sheets were checked against 1982 Daily Ambient 
Air Concentration Listings (MOE, 1984). This involved comparing the 
data recorded on the field sheet to the APIOS published data and check- 
ing for transcription errors and omissions. 

Findings - Transcription errors were not found. In all cases, 
the information listed in the database complemented the field sheets. 
Only one instance of a data point omission was found in the published 
report. A field sheet and LIS output for a sample collected at Fernberg 
(February 9-10, 1982; #301391 was found but the corresponding concentra- 
tion listings were not reported, 

3. Check - Summary of field and office comments. This involved 
tabulating the office and field comments for all daily air concentration 
measurement sites for the period January 1 to December 31, 1982. This 
table provides a quick overview of the types of comments which are being 
appended to analysed samples. Frequently, recurring field or office 
comments can sometimes be good indicators of sample contamination, poor 
instrumentation or non representative siting. 

Finding - Table 3.7 summarizes the frequency of comment codes. 
Dorset had the greatest number of sampler malfunctions 35 of 334 samples 
(10%) and abnormal sampling periods 16 of 334 samples (5%). However, 
the majority of these malfunctions did not affect the samples. Since 
the Dorset technologist is on-site, any sequencing problems were always 
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TABLE 3.7 

Frequency of Field a nd Office Conments - Daily Dry Data 

— ^* — ^ vm 



site 


No. 

of 

Records 


% 

Data 

Conplete- 

ness 

































Lab Comnient 


Field comments 


ui 1 i(,e 

Comments 


Result Remark Code 


A 


B 


C 





E 


F 


G 


H 


I 


J 


K 


F 


P 


Z 


X 


* 


> 


< T 


< W 


A 


U 


P 


< P 


Longwoods 


343 


94 


3 






1 






2 


3 


9 


1 


2 


2 




6 


3 


86 


71 


4 


11 




18 






Dorset 


334 


91 


35 








Z 




5 


I 


6 


35 








16 




21 


16 


58 


122 




e 






Charleston 
Lake 


340 


93 


lb 


1 




6 






7 








1 


2 




6 




36 


51 


21 


47 




23 






Fernberg 


317 


87 


6 


5 


1 




3 








2 






5 




3 




28 


a 


52 


151 




27 







Field Comment Code Index 

A - Sampler Malfunction 

B - Known Hydro Failure 

C - Suspected Hydro Failure 

- Known Filter Pack Leak 

E - Suspected Filter Pack Leak 

F - Gas Meter not equal to Rotameter 



G - Flow Line Problems 

H - Known Contamination 

1 - Suspected Contamination 

J - Heavy Dew or Fog 

K - Sample not Submitted 



I 

4 



o 



^^• 



Office Comment Code Index 

F - Data invalidated - Flow volume rate less than 10.000 1 per day 

P - Passive missing - Average passive results used as blank correction 

2 - Abnormal sampling period 

X - Sample lost 

Result Remark Code Index 

> - Actual result greater than value reported 
< - Actual result less than value reported 
<T - Actual result less than criterion of detection 
<W - No response, minimum possible result reported 
A - Approximate value 
Unreliable result 
Not corrected for passive 
Not corrected for passive - Reported value is a detection limit 



U 

P 

<P 



* - No data reported 
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fixed before a new pack began sampling. Charleston Lake had the second 
highest record of these occurrences, S% (sample malfunctions) and 25« 
(abnormal sampling periods). Fernberg reported the largest number of 
Invalid data due to insufficient sample volume (<1000 litres per day) 
2%. 

To determine the frequency of occurrences per site due to 
hydro failure, contamination or flow volume problems, field comment 
codes associated with each of these problems were combined and a 
percentage calculated. The results are tabulated in Table 3.8. 

4. Check - Data Validation Procedures. This involved determining 
if the procedures for validating daily dry data described in Kirk (1984) 
have been correctly executed. These procedures involve qualifying 
sample records by three methods. 

1. Field comments - which are appended to the sample records at 
the point of collection to reflect sample quality and possible 
sources of contamination. 

2. Office comments - which are appended to sample records to 
indicate failure of specific integrity tests or sample 
representativeness. 

3. Validation flags - which are appended to individual analytical 
results if outliers are detected or gross limits fail. 
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TABLE 3.8 

Frequency of Field Comments Per Site for Hydro Failure. Flow Volume 

Problems and Contamination 
Event Air Sampling Network (1982) 



Site 




Hydro Fai 

B+C 


lure 


Fl 


ow Volume 
D+E+F+G 

<1 


Contamination 

H+I 


Longwoods 







3 


Dorset 











<2 


2 


Charleston 


Lake 


<1 






4 





Fernberg 




<2 






<1 


<1 
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Field comment codes appended to samples from Longwoods and 
Fernberg during the study period (e.g. February, March, June and July, 
1982) were investigated to determine if the appropriate office comments 
and validation flags had been applied. Field data sheets, LIS outputs 
and reported data concentrations were compared. Samples which exceeded 
the gross limits were tabulated and the data examined. 

Findings - The procedures for qualifying data were for the 
most part correctly executed. Some modifications may be required when 
reporting data with the following field codes: I (suspected contamina- 
tion) and H (known contamination). When a single filter (Nylon, Teflon 
or Whatman 41) is potentially contaminated while being unloaded due to 
improper filter handling techniques (e.g. dropped on floor, touched by 
technologists bare hand, etc.) a "result remark code" should be appended 
to the analytical result associated with the analysis of that filter 
rather than to the entire sample, i.e. for nylon filter a remark result 
code should be appended to the N-HNO3 result. This also applies to a 
single filter with the code H (know contamination) appended to it. 
Table 3.9 summarizes the field comments investigated. 

Tables 3.10 and 3.11 summarize samples in which a parameter 
exceeded the upper or lower gross limit. The following observations 
were made: 



G )nt < )icl ^L lenliik C( )n^ >' <it n )i i 



- 3.38 - 

TABLE 3.9 
D»tj Qualification Check 



Site 


Field 
Hunber 


Field 
Conment 


Office 

Cwmtents 


Result 
Reirart 


Auditors Comnents 


Femberg 


30162 


A 


Z 




Sampler malfunction, two day sample, 

March 7-9, 1982 

Sample volume • 50040 litres. All 

appropriate codes and conments 

appended. 


95480 


I 






Suspected filter contamination. Teflon 
filter dropped nhile unloading. This 
was a passive filter - analytical 
results from LIS N-MH4* - !EF (egulp- 
mnt failure) N-NO, * .25 <H, SOa - 
1.25 <y. 

All appropriate codes and comments 
appended. 


90152 


E 






Suspected filter pack leak. Sample 
from March 13 08:00 hrs. to March 14 
09:00 hrs 1982. 


Longwoods 


15342 
15343 
15345 
15362 

15364 






U 


Suspected contamination of nylon 
filters all N-mOj results should be 
flagged to reflect this. 


15347 


H 


I 




Two day sample, February 14-16, 1982 
known containl nation of nylon filter - 
not flagged as unreliable. K-hnOj 
result should be flagged. 


15352 


G 






Technologists comnent on field sheet - 
"twist lock nozzle cracked at inlet 
port - Incorrect volume reported 23920 
litres". Appropriate comments 
appended. 


15501 


D 






Technologist coiiment on field sheet - 
Tygon tubing separated at coupler 
Joint - volume reported Is actual 
sample volune". Appropriate conments 
appended. 
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TABLE 3.10 

Gross Limit Checks - Lower limit 

Feb., Mar., June, July 1982 
Fernberg and Longwoods 









Concen- 














Samp 1 e 


tration 


Result 


Field 


Office 




Site 


Parameter 


Number 


u9 m-^ 


Remark 


CoiTunent 


Comment 


Auditor's Comment 


Fernberg 


SO2 


30165* 


0.00 








P>A 


; 




30152 
10577* 


II 








P>A 

P>A 


•■ 


It 


95433* 
95441* 
95449* 


II 








P>A 
P>A 
P>A 


■I 


■■ 


95455 


" 








P>A 


» 


'■ 


95457* 


II 








P>A 


V 


" 


95459 


'■ 








P>A 


■I 


" 


95461 


" 








P>A 


II 


" 


95465* 


" 








P>A 


,^ 


M 


95473* 


II 








2nd Passive Value UI! 


II 


II 


95481* 


M 








A>P 


M 


" 


95482 










LIS not available 
to auditor 


II 


II 


95489* 


II 








P>A 


II 


II 


15471* 


" 








P>A 


Longwoods 


II 


15475 


" 








P>A 


i» 


" 


15487 


" 








P>A 


II 


H 


15501* 


" 




D 




1st Passive Value UI! 


'■ 


M 


15512 


" 








P>A 


" 


" 


15517* 


" 








P>A 



* = Port # 1/8 
UI! = no data, undetermined Interference (laboratory code) 
P>A * passive result greater than active 
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TABLE 3.10 (cont'd) 
Gross Limit Checks - Lower Limit 

Feb.. Mar., June, July 1982 

Fernberg and Longwoods 









Concen- 














Sample 


tration 


Result 


Field 


Office 




Site 


Parameter 


Number 


ug fn"' 


Remark 


Comment 


Coirment 


Auditor's Comment 


fernberg 


H-NO3 


3Q152 


a. 00 


<T 








" 


It 


30133* 


" 








<U on LIS output P>A 


" 


H 


30135 


" 


<T 








II 


1. 


30136 
95425* 


IP 


<7 






P>A 


n 


I< 


95433* 


" 


<T 








" 


II 


SOIZI* 


" 


<W 








Longwoods 


H 


15352 


0.00 




G 




<T on LIS output 


■1 


*t 


1S353 


" 








<W on LIS output 


II 


n 


15354 


" 








P>A 


II 


.. 


15355 


" 








P>A 


II 


II 


15358* 


H 








P>A 


Fernberg 


N-HNO3 


95425* 


II 








P>A 


II 


" 


95426 


» 








<T on LIS output 


Longwoods 


S0_ = 


15471 


■' 








Should be 0.23 ug m"- 


" 


ir 


15358* 


II 








P>A 


II 




15496 


" 








P>A 



* = Port * 1/8 

<T = reported at the detection limit 
<W = no response, minimum possible result recorded 
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TABLE 3.11 






Gross Limi 

Feb., Ma 
Fernb 


t Checks - Upper Limit 




r., June, July 1982 
erg and Longwoods 








Concen- 








■ sue 

JB 


Parameter 


Sample 
Number 


tration 

ug m-3 


Result 
Remark 


Field 
Comment 


Office 

Comment 


Auditor' s Comment 


^rnberg 
Hi 


N-MO3 


95428 
95460 


0.09 
0.26 










n 


SO2 


30174 


16.96 










■ngwoods 
II 


50. 


15497 
15347 


31.85 
14.93 


U 


H 


2 


Nylon filter-H 


H 


II 


15503 


40.93 












II 


15344 
15345 


9.057 
9.426 


> 
> 


I 




Nylon FiUer-I 


H 


II 
II 


15346 
15347 


9.509 
4.744 


> 
> 


H 


Z 


Nylon Filter-H 


1 
II 


II 
II 


15348* 
15362 


8.944 
9.140 


> 

> 


I 




Nylon Filter-! 


*ft: Port # 1/8 
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• If an active result with a remark code is less than a passive 
result, the active is reported as "0" and the remark code is 
removed. However, for samples 15352, 15353 and 95426 the 
result remark codes (<T, <W1 appended to analytical results on 
the L.I.S. output did not appear in the Daily Ambient Air 
Concentration Listings (MOE, 1984). 

• With the exception of one instance, all exceedances of the 
upper limits were not flagged as unreliable. It should be 
noted that all exceedences of the upper limits were studied by 
a staff scientist. 

• For all instances where the passive result was equal to or 
exceeded the active result a zero was reported. 

5. Check - Passive Correction Calculations. This involved deter- 
mining if the passive correction calculations were executed properly. 
Filters in sampling ports 2 to 7 are corrected using the equation shown 
below. Twenty-five percent of data from the study period corrected 
using this calculation was spot checked. 



^ , r. Active Result -6/7 Passive Result ( ug fil" ) 
Reported-Cone. = ^^^ ^ ^ 

(ug m* ) Sample volume (m ) 



Switched filters (ports 1 and 8) are left in the field for a 
two-week period and are corrected using the equation shown below. All 
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switched filters from the study period were examined for passive correc- 
tion errors. 



Active Result - P^ - 6/7 ?^ (yg fil* ) 



Reported^Conc. = — J7 

(yg m' ) Sample Volume (m ) 



where: 



p = Passive Result from week 1 
Pp = Passive Result from week 2 



If a passive result is: 

1. reported at the 'detection limit (<T) then a value correspond- 
ing to one half the detection limit is utilized for passive 
correction. 

2. reported as no response, minimum possible result reported (<W) 
then a value of zero is utilized for passive correction. 

3. equal to or exceeding the active result then the concentration 
is reported as equal to zero. 

Findings - A few discrepancies (four samples) between the 
reported concentration and calculated concentration for switched filters 
were observed and tabulated (Table 3.12). The samples appear to have 
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TABLE 3.12 
Calculation Errors - Switched Filters 



7i 

1- 











1st 


2nd 








Sample 


Volume 


Parameter 


Active 


Passive 


Passive 


Calculated 


Reported 




No. 


ni3 




Result 


Result 


Result 


Cone. 


Cone. 


Comments 








vg fil-^ 


ug fil-^ 


ug fil-^ 


ug m-3 


ug m~^ 




30155 


23.58 


N-NO3 


1.05 


.25 < W 


.25 < T 


0.04 


0.06 


associated 






j 










passive 


















numbers not 


















recorded 
on field 
data sheets 


15470 


26.36 


N-NH^ 


5.10 


0.70 


.25 


0.159 


0.171 


only 1 
















associated 






N-HNO3 


8.25 


5.25 


2.50 


0.03 


0.14 


passive 
















number 






SO, 


60.05 


60.05 


33.35 


0.00 


0.33 


recorded 


















on field 
data sheet 


15478 


26.37 


so,= 


170.00 


1.25 <W 


1.25<W 


6.41 


6.45 


II 






N-HNO3 


58.75 


2.50 


5.00 


1.97 


2.15 








SO, 


247.00 


33.55 


56.70 


6.25 


8.29 




15993 


18.63 


so,= 


335.00 


1.25 


3.75 


18.82 


18.88 


II 






N-HN03 


25.20 


2.50 


4.20 


1.04 


1.16 





I 



^ 

*» 
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been corrected using the wrong passive filter results. Field data 
sheets for passive and active samples were examined and the associated 
passive sample numbers for these switched filters were determined. 
These "switched" errors were found to have originated from the computer 
program which assigns passive corrections to active results. Because 
these four samples had "unusual" passive dates, the errors went un- 
noticed. The error would have been eliminated if the passive numbers 
themselves, recorded on the field sheets, were used rather than passive 
dates (presently being used in the networks for validation purposes). 

While performing passive calculation checks, it was noted that 
in some instances an average passive value was determined and the active 
concentration calculated and reported (Table 3.13) when: 

a) a passive filter result was missing due to laboratory equip- 
ment failure or undetermined interference; or 

b) one of the passive filter results exceeded the active filter 
result; or 

c) a passive sample was not received at the laboratory. 

This procedure, which was carried out at the discretion of the database 
manager, should be documented and included in the Data Management 
section of the APIOS Technical and Operating Manual. When reporting 
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TABLE 3.U 
Blank Corrections Using an Average Passive Value 



Sample 
No. 


Volume 

m3 


Parameter 


Active 

Result 

vig fil*' 


1st 
Passive 
Result 
ug fil-i 


2nd 
Passive 
Result 
ug fil-i 


Reported 

Cone. 

pg m'^ 


Comments 


95473 


22.34 


N-NH/ 


2.30 


0.05<T 


!EF 


0.100 


!EF = laboratory equipment 
failure 


15374 


27.72 


M-NH/ 


50.00> 


!EF 


!EF 


1.774> 


!EF = laboratory equipment 
failure 


15390 


28.75 


N-NH4-^ 


48.50 


!EF 


0.75 


1.649 


!EF = laboratory equipment 
failure 


15390 


28.75 


so.= 


570.00 


30.00 


NR 


18.19 


NR = laboratory did not 
receive sample 


15342 


28.22 


SO2 


420.00 


20.00 


580.00 


13.17 




15348 


27.74 


SO2 


23.30 


580.00 


13.30 


19.96 




15493 


18.63 


SO2 


220.00 


76.70 


UI 


9.51 


UI = undetermined 
interference 



■ 
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these data the office cornment code "P" (passive filter missing - average 
passive result used as blank correction) or an equivalent result remark 
should have been appended. 

6. Check - Passive results for the Longwoods and Fernberg sites 
were tabulated by season for the study period. The mean, standard 
deviation, range and variance were calculated for each and the data 
examined. 

Findings - (see Tables 3.14 to 3.17] Results with an asterisk 
{*) indicate that the passive result was greater than an associated 
active result. The passive SO2 results are extremely variable and all 
SOp passive results from Fernberg during June and July (1982) were 
greater than the active results. 

Samples No. 15389 and 15397 have SO^ results reported as 0.25 
ug filter"-^ which is less than the detection limit of 1.25 ug filter . 

The variability of passives results should be determined and 
accounted for in the calculation. 

3.4 Performance Audit of Data Handling and Validation Procedures - 



Cumulative (28-day) Air Concentration Data 

Specific data validation procedures have been outlined (Chung, 

1984) to screen cumulative ambient air monitoring results. These proce- 
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TABLE 3.14 
1982 Passive Results for Longwood (June, July) 



Passive 

ID 


Date 
Off 


Associated 
Active 


vjg fil-i 


NO, 
ug fil-^ 


ug fil"^ 


HNO3 
ug fii-1 


SO, 
ug fil-i 
(Nylon) 


S0„ 

ug fii-i 


Cottmient 


Detection Limits 




0.05 


0.25 


1.25 


0.25 


2.5 


3.35 




15469 


2/6/82 


15462 to 
15470 


.70 


.25<W 


1.25<W 


5.25 


36.25 


60.05* 


*P > A 


15477 


9/6/82 


15470 to 
15478 


.25 


.25<W 


1.25<W 


2.50 


22.50 


33.55 




15485 


15/6/82 


15478 to 
15486 


.25 


.25<W 


1.25 


5.00 


22.50 


56.76 




15492 


22/6/82 


15486 to 
15493 


.10 


.25<W 


1.25 


2.25 


21.20 


76.70 




15500 


13/7/82 


15493 to 
15501 


.10<T 


.25<W 


3.75 


4.20 


31.20 


!UI 


undetermined 
interference lUI 


15508 


20/7/82 


15501 to 
15509 


.40 


.25<W 


1.25 


3.00 


27.50 


53.40 




15516 


27/7/82 


15509 to 
15517 


.05<T 


.25<W 


1.25<W 


2.50 


16.20 


103.00* 


* P > A 






Mean 


0.26 


.25 


1.61 


3.53 


25.33 


63.91 








Std 
Deviation 


0,23 





0.95 


1.26 


6.77 


23.62 








Variance 


0.05 





0.89 


1.60 


45.80 


55 








Range 


0.65 





2.50 


3.00 


20.05 


69.45 





00 



1982 Passive Results for Longwood (February, March) 



n, 



Passive 

10 


Date 
Off 


Associated 
Active 


wg ni-^ 


NO, 

ug fii-i 


SO, 

ug fii-^ 


HNO3 
\>9 fil"* 


,ig fil-i 
(Nylon) 


SO2 

pg fu'^ 


Cominent 


Detection Limits 




0.05 


0.25 


1.25 


0.25 


2.5 


3.35 




15341 


9/2/82 


15334 to 
15342 


.25<T 


.25<T 


2.50 


5.25 


28.80 


20.00 




15349 


16/2/82 


15342 to 
15350 


.25<T 


.50 


2.50 


8.25 


32.50 


580.00* 


*P > A 


15357 


23/2/82 


15350 to 
15358 


0.45 


1.75* 


8.75* 


2.75 


37.50 


13.30 


*P > A 


15365 


2/3/82 


15358 to 
15366 


!EF 


.25 


1.25 


1.25 


20.00 


10.00 




15373 


9/3/82 


15366 to 
15374 


!EF 


1.00 


3.75 


3.50 


28.80 


26.70 




15381 


16/3/82 


15374 to 
15382 


!EF 


.50 


1.25<T 


3.00 


36.20 


33.30 




15389 


23/3/82 


15382 to 
15390 


!EF 


.25<T 


* .25<T 


2.50 


28.80 


30.00 


*less than de- 
tection limit 


15397 


1/4/82 


15462 to 
15470 


.75 


.25<W 


* .25<W 


3.00 


27.50 


!NR 


♦less than de- 
tection limit 






Mean 


0.42 


0.59 


2.56 


3.69 


30.01 


101.9 








Std 
Deviation 


0.23 


0.53 


2.77 


2.15 


5.50 


211 








Variance 


0.05 


0.28 


7.69 


4.63 


30.21 


445 








Range 


0.50 


1.50 


8.50 


7.00 


17.50 


570.00 





I 



IJ3 



TABLE 3.16 
1982 Passive Results for Fernberg (June, July) 



n 



'<: 



Passive 

ID 


Date 
Off 


Associated 
Active 


NH, 
ug fil'^ 


NO3 
ug fil-i 


so, 
pg fii-i 


HNO3 

ug fii"^ 


SO, 
yg fil-i 
(Nylon) 


1 , — — 

S0„ 

pg fil-i 


Comment 


Detection Limits 




0.05 


0.25 


1.25 


0.25 


2.5 


3.35 




95432 


8/6/82 


95425 to 
95433 


.05<T 


.25<T 


1.25<T 


1.00 


3.75 


50.00* 


* P > A 


95440 


15/6/82 


95433 to 
95441 


.05<T 


.25<T 


1.25<T 


.25<T 


1.25<T 


63.40* 


* P > A 


95448 


22/6/82 


95441 to 
95449 


.05<T 


.25<W 


1.25<W 


.25<W 


1.25<W 


5.67* 


* P > A 


95456 


29/6/82 


95450 to 
95457 


.05<T 


.25<W 


2.50 


.75 


3.75 


66.7* 


* P > A 


95464 


6/7/82 


95457 to 
95465 


.05<T 


.25<W 


1.25<W 


.50 


2.50 


53.40* 


* P > A 


95472 


13/7/82 


95465 to 
95473 


.05<T 


.25<W 


1.25<W 


.50 


1.25<T 


40.00* 


* P > A 


95480 


20/7/82 


'95473"to 
95481 


!EF I 


.25<Wi 


1.25<Wl 


.75 


3.75 


!UI 


1 Teflon 
filter dropped 


95488 


27/7/82 


95481 to 
95489 


.10<T 


.25<W 


1.25<W 


.25<W 


2.50 


30.00* 


* P > A 


95496 


5/8/82 


95489 to 
95497 


.10<T 


.25<W 


1.25<W 


1.00 


2.50 


23.30* 


* P > A 






Mean 


0.0625 


.25 


1.38 


0.58 


2.50 


47.94 








Std 
Deviation 


0.023 





0.42 


0.31 


1.08 


15.55 








Variance 


0.005 





0.174 


0.09 


1.17 


242 








Range 


0.05 





1.25 


0.75 


2.50 


43.40 





I 

CO 

01 
o 



- 3.51 - 

TABLE 3.17 
1982 Passive Results for Fernberg (February. March) 



Passive 
ID 



Date 
Off 



Detection Limits 



30132 2/2/82 



Associated 
Active 



30130 



NH. 



NO, 



ug 



fil-i 



0.05 



.20 



0.25 



,25<T 



SO, 
ug fir» 



1.25 



1.25<T 



HNOj 
u9 fil 



-1 



0.25 



SO. 
ug fil-^ 
(Nylon) 



2.5 



1.50 



11.20 



SO, 
ug fil 



-1 



3.35 



10.00 



Comnent 



30117 



9/2/82 



30134 to 
30118 



.20 



25<T 1.25<W 



.75 



6.25 



13.30 



30180 



23/2/82 



30173 to 
30165 



.05<W 



.25<W 



1.25<W 



25<W 20.00 



26.70* 



* P > A 



30172 



2/3/82 30165 to 
30171 



.15 



.25<W 



1.25<:W .75 



10.00 16.70 



30164 



9/3/82 



30157 to 
30163 



.25 



.25<W 



1.2S<W 



1.00 



8.75 



13.30 



30156 



16/3/82 



30149 to 
30157 



.30 



.25<T 



1.25<T 



.75 



15.00 



70.00 



30140 



23/3/82 



30133 to 
30138 



.30 



.25<T 



1.25<W 



.25 



10.00 



53.40 



10684 



30/3/82 



10678 to 
10683 



.05<T 



.25<W 



1.25<W 



.75 



30.00 



277.00* 



P > A 



Mean 



0.19 



.25 



1.25 



0.75 



13.90 



60.05 



Std 
Deviation 



0.10 



0.40 



7.75 



90.30 



Variance 



0.01 



0.161 



60.0 



8154 



Range 



0.25 



1.25 



23.75 



257.00 
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dures consist of 1) an integrity check of sample analysis and 2) statis- 
tical screening of the data for outliers using the Dixon Ratio Test. 

In the performance audit non-routine manual checks were 
performed to determine if the data validation procedures were being 
utilized properly. The audited data set consisted of data collected at 
five sites (one site in each of the five regions); Gowganda, Shallow 
Lake, Uxbridge, Pickle Lake and Smiths Falls for the period January to 
December of 1982. A description of the checks performed along with the 
findings are described below. 

1. Check - Field sheets were compared to operator sheets to 
ensure that all information recorded on the operator sheets was trans- 
ferred and that it was transcribed correctly. The field sheets were 
also checked for proper changeover dates and proper use of field codes. 

The field sheets contain all information pertinent to the 
samples submitted such as the sampling period, filter type, any problems 
observed, volume readings, etc. Field and operator sheets were avail- 
able for all sites with the exception of Pickle Lake (operator sheets 
were unavailable). 
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Findings - A summary of the discrepancies found between the 
field and operator sheets is shown on Table 3.18. In most cases the 
discrepancies are due to errors in information transfer or missing 
information. It is recommended that the technologists be made aware of 
these types of discrepancies and that a system of automatic verification 
te introduced which would prevent these inaccuracies. This system was 
also recommended in the Phase I audit (CSC, 1984) and involves comparing 
a portion (10%) of the operator and field sheets every six months. This 
would be the responsibility of the QAC. When a discrepancy is found, 
the technologist is made aware of it and a revised field sheet 
submitted. 

2. Check - Reported data against field sheets. The reported data 
was checked manually against the field sheets to ensure all data were 
transcribed correctly. Field sheets and reported data were available 
for all of the five sites and data for 60 samples were examined. 

Findings - A summary of the discrepancies found between the 
field sheets and reported information is found in Table 3.19. For the 
most part, information transferred from field sheets to the database was 
correctly executed with the exception of the data from the Uxbridge site 
(12 transcription errors were noted). A system of automatic 
verification should be implemented to minimize this type of problem. If 
complete and accurate data are to be reported, transcription errors and 
omissions must be eliminated. The QAC should continue to spot check 
field sheets and database output particularly from the Uxbridge site. 
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TABLE 3.18 
Discrepancies Between Field and Operator Sheets 







Number of Occurrences 






Information 










- - 




Not Trans- 
ferred From 






Improper 
Code Used 






# of Data 


Operator 


Information 


Sample 


for Filter 


Filter Lot 




Sheets 


Sheets to 


Transferred 


Changed on 


Type or SIS 


Number Not 


Site 


Examined 


Field Sheets 


Incorrectly 


Wrong Day 


Sample Type 


Recorded 


Gowganda 


12 


8 


5 


4 


1 


10 


Uxbridge 


14 


2 


1 


7 


3 


11 


Shallow 


12 


4 


1 





1 





Lake 














Smiths Falls 


14 


4 





3 


1 


10 



Note: Pickle Lake excluded; operator sheets were not available. 
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TABLE 3.19 
Discrepancies Between Field Sheets and Reported Data 







Number of Occurrences 






Volume not 


Field Codes 


Information 


Incorrect 






Recorded 


not Trans- 


Transferred 


Time Trans- 








ferred from 


Incorrectly 


cribed on 




# of Data 




Field Sheet 


From Field 


Database 




Points 




to Database 


Sheets to 


Output 


Site 


Examined 






Database 




Gowganda 


12 











1 


Uxbridge 


12 





1 


12 




Shallow 


12 





1 








Lake 












Smiths Falls 


12 


1 








1 


Pickle Lake 


12 





1 


3 


3 
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3. Check - Summary of field and office comments. Field, office 
and laboratory comments for cumulative ambient air monitoring network 
for the period of January to December 1982 were examined. 

Findings - Table 3.20 summarizes the frequency of field, 
office and lab comments. To determine the frequency of occurrences per 
site due to hydro failure, sample contamination or flow volume problems, 
field comment codes associated with each of these problems were combined 
and a relative frequency was calculated. The results are tabulated in 
Table 3.21. 

4. Check - Data Validation Procedures. Procedures outlined in 
Data Validation Procedures, Cumulative Ambient Air Monitoring Network 
(Chung 1984) were examined to determine whether they are being implemen- 
ted properly. Validation procedures utilized consist of two approaches. 
The first approach consists of an integrity check in which the sample 
flow, volume and sampling period are examined and the second is a 
statistical screening of the data for outliers (using the Dixon Ratio 
tests). 

All reported data from the five sites selected during the 
period January to December 1982 were screened to ensure proper tags 
(i.e. Z for sampling period, F for flow volumes) were applied to 
samples that were not within specified guidelines and 25^ of the 
reported data were randomly checked using the Dixon Ratio test. 

O Jilt <)(■(! S irniii!< i. oirf "'■■■'>' •''" 



- 3.57 - 

TABLE 3.20 
Frequency of Field and Office Consents - low Vol Data (1982) 



Site 


Number of 
Records 






















1 








1 ■ 1 

Lab Comments 


Field Comments 


Office 
Comments 


Result Remark 
Code 


A 


B 


C 





E 


F 


G 


H 


I 


J 


K 


F 


Z 


X 


* 


> 


<T 


<W 


A 


U 


Gowganda 


12 


























4 




5 


1 








1 


Uxb ridge 


12 






1 




















2 




11 




1 


2 




3 


Shallow 

Lake 


12 


3 






1 




2 




1 


3 






1 


1 




22 




1 








Smiths Falls 


12 


1 


3 












2 


2 






2 


3 




30 


1 








2 


Pickle Lake 


12 


1 


1 










5 










3 


6 




23 


2 


4 


1 




1 



G - Flow Line Problems 

H - Known Contamination 

I - Suspected Contamination 

J - Heavy Dew or Fog 

K - Sample Not Submitted 



Field Cormient Code Index 

A - Sampler Malfunction 

B - Known Hydro Failure 

C - Suspected Hydro Failure 

D - Known Filter Pack Leak 

I - Suspected Filter Pack Leak 

F - Gasmeter Not Equal to Rotameter 

Office Comment Code Index 

F - Data invalidated - flow volume not within upper and lower limits (150% and S0% of 
2880 ntres/day respectively) i- i* co 

Z - Sampling period abnormal - ■sampling period not within upper and lower limits [23 
and 27 days respectively) 

X - Sample lost 

Result Remark Code 

> - Actual result greater than value reported 

< - Actual result less than value reported 

<T - Actual result less than crUerion of detection 

<W - No response, minimum possible result reported 

A - Approximate value 

U - Unreliable result 

* - No data reported 
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TABLE 3.21 

Frequency of Field Comments Per Site for Hydro Failu re, Flow Volume 

Problems and Contamination 
Cumul ati ve Air Sampling Network (1982) 



Site 

Gowganda 
Uxbridge 
Shallow Lake 
Smiths Falls 
Pickle Lake 



Hydro Failure 
B+C 



8 

33 
25 

17 



% 

Flow Volume 
D+E+F-i-G 





25 

17 
33 



% 
Contamination 

H+I 



33 

33 
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All samples had the proper office comment codes appended. 
Samples which exceeded the upper and lower limits for flow volume (150% 
and 50% of 2880 litre/day, respectively) were appended with the code 
"F", Samples which exceeded the upper and lower limits of the standard 
sampling period (29 and 27 days, respectively) were appended with the 
code "Z". 

The Dixon Ratio test was applied to approximately 25% of the 
data from January to December 1982. It was found that it is being used 
correctly to identify outliers in the reported data. 
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4. SYSTEMS RELATED AUDIT 

A systems related audit involves reviewing network operations 
and systems to determine how well they satisfy the objectives of a moni- 
toring network. The primary objectives of the APIOS monitoring network 
are the quantification of both wet and dry deposition patterns across 
the province of Ontario (Chan et al., 1984, 1st revised edition). 

Since the Phase I systems audit reviewed, in detail, the over- 
all operating system and QA system of the APIOS network, the Phase II 
system audit concentrated on an evaluation of the daily air concentra- 
tion data reporting and validation procedures, recent changes to the 
network organization, and routine QA/QC procedures with respect to QA 
objectives. 

4,1 Daily Air Concentration Data Validation Procedures and 
Reporting 

The APIOS data validation procedures for the daily ambient air 
monitoring network have recently been documented (Kirk, 1984) and should 
be included in the Data Management section of the APIOS Technical and 
Operating Manual (Bardswick, 1983). 

Basically, these procedures involve two editing/screening 
stages. The first stage involves comparing the database output, in 
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which omissions or inconsistencies have been noted (e.g. missing sample 
volumes, abnormal sampling periods, dates, times, etc.), to the field 
data sheets. This step ensures completeness and accuracy of the data 
set. The second stage utilizes statistical computerized screening tech- 
niques such as gross limit checks and principal component analysis to 
select unlikely analytical results. These selected results are then 
reviewed by the database scientist who subjectively decides whether 
results appear reasonable or not. The data validation system seems to 
be well conceived and Implemented, however, modifications are suggested 
in qualifying and reporting data. 

Use of Comment Codes 

As discussed in the performance audit, data with the field 
comment codes I (suspected contamination) and H (known contamination) 
often provide misleading information about the quality of the data set. 
This has been noted when the codes apply to one filter in particular as 
opposed to the entire set of filters comprising one sample. When this 
occurs, an appropriate result remark code should be appended to the 
analytical results of the filter rather than appending the field comment 
code to the entire data set for the sample. 

Inconsistencies in the use of these codes has also been noted 
where one technologist has used the code I, while another uses the code 
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H for the same situation, e.g. "dropped nylon filter on floor while 
unloading filter pack". 

The use of these codes (H and I) should be standardized. Any 
observations which could affect the integrity of any filters in the 
filter pack should always be included on the field data sheets. These 
observations have proven to be quite valuable to the database scientist 
when qualifying data with field comment codes appended. 

The letter "P" is utilized as an office comment code and a 
result remark code. However, the definitions associated with these 
codes differ. The office comment code "P" indicates that an average 
passive result was used for blank correction and the result remark code 
"P" indicates that a sample was not corrected with a passive result. 
Because these codes have different meanings the same letter should not 
be used. It should be noted, however, that for the 1982 data the office 
cotment code "P" was never used in the data validation procedure. 
Therefore, it should have been removed from the 1982 report which 
describes these codes. 

Symbols utilized as office comments codes, field comment 
codes, and result remark codes should be unique and have precise 
definitions to avoid ambiguity in interpretation. 
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When a passive result 1s not available because of laboratory 
equipment failure or undetermined interference, an average passive 
result is utilized for passive correction. When this occurs for a 
particular parameter in the data set, a result remark code equivalent to 
the office comment code "P - passive missing, average passive result 
used for blank correction" should be appended. 

The procedures or guidelines utilized by the database 
scientist in determining whether an active result will be corrected 
using an average of passive values should be documented and included in 
the Data Management section of the APIOS Technical and Operating Manual 
(Bardswick, 1983), as well as Data Validation Procedures (Kirk, 1984). 

Reporting Data 

Currently, the APIOS network reports data in the form of two 
or three significant figures for air concentration measurements. The 
data are calculated by subtraction of a weekly passive filter value from 
the corresponding actively sampled integrative value for each parameter. 
Active values less than or equal to the passive value are reported as 
zero. 

The performance audit section of this report reveals that in 
many instances the passive result was greater than the active result. 
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it Is recommended that re-evaluation of passive correction is in order 
to determine why passive results are frequently greater than the active 
results. 

The reporting of data should ensure that results are not 
arbitrarily truncated nor set to zero if active filter results give 
negative numbers after correction for passive filter contribution. The 
level of confidence with low level data should be reported to data users 
in the form of codes attached to specific numbers outlining measurement, 
method detection criteria and detection limits, and the statistical 
levels of confidence in numbers. (See the IJC Bulletin on detection 
limits, Appendix III.) 

4.2 Network Organization and Operation 

The recent change to regional ization of field operations has 
an important impact on the quality of the data collected and produced by 
the network. From an operational point of view, it will ease logistical 
constraints of field operations by allowing the APIOS technical people 
access to other ministry resources. It should also improve many aspects 
of the field program by providing additional support to APIOS field 
activities, particularly in the remote areas of the province because 
other ministry employees will be able to assist in sample collection and 
maintenance. 
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However, APIOS and regional managers must take appropriate 
action to ensure that APIOS technical staff concentrate their efforts on 
the APIOS programme to ensure continuity and comparability within and 
between regions. 

Reduction in the technologists travelling time (by co-ordina- 
ting site visits with other ministry activities) should improve staff 
morale and allow technologists to spend time on other aspects of the 
APIOS network such as processing and interpretting QA/QC information and 
data validation procedures. Regional ization will require work on the 
part of the APIOS head office staff in terms of support, organizational 
requirements, co-ordination and quality control. 

The emphasis on network QA/QC must continue. QA/QC and over- 
all network operating systems must improve in order to implement the 
proper QA/QC monitoring procedures to ensure the system is in control. 
It will be the responsibility of the QA Coordinator to ensure that the 
field personnel are meeting their APIOS field requirements by designing 
and implementing a system to monitor field QA/QC procedures. In 
particular, his functions with respect to corrective action monitoring, 
routine QA/QC assessment and QA/QC reporting and interpretation must 
expand significantly. 

A sound system for co-ordinating APIOS activities in all 
regions should be employed. Regional managers (Air Quality Chiefs) 
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should be involved in the relevant aspects of field programme planning, 
and a mechanism should be implemented for incorporating their input into 
the network operating system. It is recommended that regional air 
quality chiefs and APIOS Network managers meet on an annual basis to 
discuss these aspects of the programme. 

Recent staff turnovers in the APIOS programme have had 
significant negative impacts on the operations of the field programmes 
in the northeastern and southeastern regions. Attempts to replace 
full-time technologists with part-time consulting personnel were "band 
aid" solutions to staffing problems that would have been effective in 
the short-term (i.e. less than one month), but were totally inadequate 
during long-term periods of eight and seven months at Kingston and 
Sudbury, respectively. Consulting personnel were asked to collect and 
process samples and "maintain" site operations in the time allocated 
{four days per week in the Kingston region and two weeks per month in 
the Sudbury region). Some problems arising from this situation 
included: 

poor communication between operators and network personnel 
which led to inadequate instrument servicing; 

missed changeovers; 

late spring instrument changeover which resulted in a loss of 
storage gauge data; 
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poor communications between operator and consultants because 
of time restraints in the field. 

Required instrument servicing was performed as quickly as 
possible to minimize downtime and loss of data, however, complete 
preventive maintenance was not carried out due to time constraints 
placed on the consultants. In order to avoid this type of problem in 
the future, it is recommended that a time limit of one month be put on 
replacement of network personnel and that network management support 
this time limit vigourously to help maintain the quality of the network 
data. 

4.3 Evaluation of Routine QA/QC Procedures with Respect to QA 
Objectives 

Routine QA/QC monitoring has now been implemented in many 
areas of the program and has been evaluated for the cumulative network 
for the period June 1980 to December 1981 (Bardswick, 1984). In this 
report information on data recovery, reasons for loss of data, labora- 
tory QA/QC and special studies were reported. 

All QA/QC procedures which have been implemented should be 
reported, and it is essential that these reports be prepared on an 
annual basis as quickly as possible, in order that the information 
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contained in them may be used in the annual network review sessions 
described in the QA Manual (CSC, 1984b). 

Since data from the recently implemented routine QA procedures 
is not yet available, it was not possible in this audit to quantify data 
quality. 
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5. RECOMMENDATIONS SUMMARY AND PHASE I AUDIT FOLLOW-UP 

This chapter summarizes, in point form, the recommendations 
made in this report (in the order in which they appear), as well as 
reviewing the APIOS actions on the recommendations from the Phase I 
audit. The section in which the recommendations originate is referenced 
in brackets. Items which may significantly affect data quality have 
been identified by an asterisk (*) and the QAC should review these 
immediately and take the appropriate action. Other recommendations 
should be acted on as soon as possible. 

* 1. Corrective action at audited sites (Secton 3.1; Appendix II). 

* 2. Review sites with a marginal or poor rating (Section 3.1). 

3. Post "Procedural short lists" at monitoring sites (Section 
3.1). 

4. Inconsistencies in instrument configuration from region to 
region should be removed (Section 3.1). 

5. Service MIC samplers at Dorset (Section 3.2). 

* 6. Update network site documentation for the sites in the Central 

Region (Section 3.2). 
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7. Regional technologists should continue to maintain close 
communication with operators, encourage operators to contact 
them for any reason and try to co-ordinate their site visits 
so that they can meet with the operators as often as possible 
(Section 3.2) . 

* 8. Technologists should carry out procedures audit at least twice 

a year (Section 3.2). 

9. Operators should fill out operator sheets, monthly site 
summaries and log-books in duplicate, one copy should be 
retained for future reference and a second copy forwarded to 
regional technologist (Section 3.2). 

* 10. Quality Assurance Activity Log Sheet should be developed and 

implemented in network operations and procedures (Section 
3.2). 

* 11. Retraining sessions for technologists should be carried out at 

the next Quarterly APIOS Technologist meeting (Section 3.2). 

* 12. Technologists should ensure that they have on hand at least 

the recomnended Instrument spare parts at all times (Section 
3,2). 
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* 13. Instruments in the Central region should be calibrated and 

serviced on a routine basis (Section 3.2). 

* 14. Technologists should visually check field data sheets for 

omissions and errors (Section 3.3.1). 

* 15. Procedures or guidelines followed by the database scientist, 

used for correcting active values and an "average passive 
value", should be reviewed and documented and an appropriate 
code appended to the reported concentration (Section 3.3.1). 

16. Procedures or guidelines used in (15) should be included in 
the Data Management section of the APIOS Technical and 
Operating Manual (Section 3.3.1). 

17. The method of correction for passive filter results should be 
reviewed (Section 3.3.1). 

18. Air concentration data validation procedures should be inclu- 
ded in the Data Management section of the APIOS Technical and 
Operating Manual (Section 3.3.1). 
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19. QAC Should spot check 10% of the operator and field sheets 
every six months to ensure accuracy and completeness of 
reported data (Section 3.4). 

20. QAC should continue to spot check field sheets and database 
output {Section 3.4). 

21. Result remark code indicating potential or known contamination 
of a particular filter should be incorporated into the coding 
system [Section 4.1). 

22. The use of the field comment codes H and I should be standard- 
ized (Section 4.1). 

23. Observations which could affect the integrity of any filters 
should always be included on the field data sheets (Section 
4.1). 

24. Symbols used for office comments, fields comments and result 
remarks should be unique and have precise definitions to avoid 
ambiguity in interpretation (Section 4.1). 

25. The level of confidence in the low level data should be 
reported to data users in the form of codes attached to 
specific numbers, outlining measurement method detection 
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criteria and detection limits and the statistical levels of 
confidence in numbers (Section 4.1). 

26. Regional air quality chiefs and APIOS network managers should 
meet on an annual basis to discuss these aspects of the 
program (Section 4.2). 

* 27. QA/QC reports should be generated for all data sets and QA/QC 
information should be included in these data sets (Section 
4.3). 

Phase 1 Audit Recommendations and the Actions Taken 

The following describes the actions taken in response to the 
47 recommendations generated from the Phase I audit findings during 
1983-84. 

#1 Corrective action at audited sites. 

action Monitoring sites that required corrective action have been 
investigated by the QA Coordinator and the environmental tech- 
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nologist responsible for the site. Where possible, changes 
have been made to improve the site {i.e. remove a tree, 
upgrade operator, etc.). A technologist meeting was held in 
which all corrective actions were discussed and what steps 
would be taken by the technologists to address the corrective 
actions recommended by the auditor. 

#2 Review of marginal or poor siting by QA Coordinator. 

action A review of all sites was undertaken by the QA Coordinator. 
Using site documentation, photographs, and all of the 1982 
data collected at each site, an evaluation of the representa- 
tiveness or non-representativeness was determined for every 
site. From this evaluation, the Milton and Attawapiskat moni- 
toring sites were removed from the networks. Colchester and 
Moonbeam, both rated "poor" by the auditors are still consid- 
ered representative monitoring sites by APIOS personnel. 

#3 Feedback to operators on networks. 

action Network operators are kept abreast of monitoring activities by 
way of a semi-annual newsletter. Also, annual reports of 
chemical listings for every site are provided to ewer)f 
operator. 
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#4 Update and upgrade training for operators. 

action All technologists have been instructed to thoroughly train 
every operator in operating procedures. As well, a training 
video is being prepared for retraining and updating purposes. 

#5 Technologists to review APIOS Technical and Operating Manual with 

operators, 
action The review of operating procedures in recommendation #4 
includes this review. 

#6 Detailed specifications of instrument checks. 

action Instrument checks are presently specified for all instruments. 
The complexity of an instrument determines the amount of 
detail required for the instrument check. If more time and 
manpower become available in the programme, detailed 
instrumentation manuals will be prepared for all 
instrumentation. However, it is felt that the present 
instrument checks are more than adequate to ensure the proper 
operation of the networks. 

#7 Sangamo/MIC sensor resistance and clutch setting should be set more 

often by technologists. 
#8 During winter months, snow and ice should be cleaned off the 

instrument housing of the Sangamo/MIC and Aerochem. 
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#9 Operators should be instructed by technologists on how to keep 

on-site log books. 
action Technologists have been instructed to carry out these 
actions. 

#10 Incorporate log book recorded information into data screening and 
validation procedures. 

action Any information recorded in the log book that is relevant to a 
sample is presently entered onto the operator sheet under "Any 
other factors affecting sample". To attempt to incorporate 
log books on a routine basis into the data screening process 
would be very difficult and time consuming. It is felt that 
if a sample's representativeness is questionable, the log book 
can be consulted to aid in the validation procedure. 

#11 Technologists should review filter handling procedures (especially 
those that could provide a potential contamination; type of 
tweezers, washing of filter packs). 

#12 Technologists should carry out routine calibration and maintenance 
of instrumentation. 

#13 Technologists should minimize the exposure time of all precipita- 
tion samples, event and cumulative, to temperatures above 4°C. 

#14 Technologists should alert head office of site changes as described 
in the Technical and Operating Manual. 
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#16 Technologists should not park vehicles near samplers. If a vehicle 
must be brought close to the sampler location, it should never be 
left running. 

action Technologists have been instructed to carry out these 

actions. 

#15 Head office should have an upgrading training session for APIOS 

technologists. 
action As soon as technologist vacancies are filled, an upgrading 
training session will be held. 

#17 Operator Sheets should be compared with Field Sheets periodically 

by QA Coordinator for errors in data transfer. 
#18 Field Sheets should be checked periodically against database for 

errors in data transfer. 
#19 QA Coordinator should periodically verify that technologists are 

carrying out storage gauge depth determinations correctly. 
action These types of checks will be carried out on a routine basis. 

#20 Technologists should be made aware of deviations in sampling dates 

and times at their sites, 
action Technologists are aware of these deviations. Technologists 

have requested operators to follow sampling dates more 

closely. 
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#21 Quality of data collected at Gowganda, Moonbeam, Attawapiskat, Ear 
Falls, and Pickle Lake may be questionable. 

action As indicated in the actions responding to recommendation #2, a 
review of all sites was undertaken. The Milton and Attawapis- 
kat sites, based on the 1982 data, were the only sites removed 
from the networks due to evidence of local contamination. 

#22 To restructure the APIOS Technical and Operating Manual into separ- 
ate manuals in order to gear for the intended users. 

action Presently, the manual is divided into major sections, with 
each section geared to the intended user. There are no advan- 
tages in restructuring the present manual. 

#23 Preparation of the QA manual. 
action A QA manual has been prepared. 

#24 Instrumentation servicing procedures for both operators and tech- 
nologists should be prepared. 

action Operators are not required to service any Instruments. If 
time and manpower becomes available, servicing procedures will 
be developed, however at present most technologists have 
sufficient experience and fundamental skills to repair and 
service monitoring equipment. 
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#25 The method currently used to fit sample bags should be discontinued 

and another method adopted, 
action The improved method, if followed correctly, will result in a 
properly placed bag in the collector. 

#26 Storage procedures for cumulative precipitation samples for 
operators and technologists should be developed and included in the 
operating procedures. 
#27 Filter handling blank procedures should be modified to include all 

potential contamination areas, 
action Storage procedures will be prepared. Filter handling blanks 
have already been modified to include all potential contamina- 
tion areas. 

#28 A schedule detailing which samples and gauges are to be used at 
different times of the year should be included in the operator 

procedures. 
action During the Spring and Fall, technologists are instructed to 
set up both types of gauges. Operators are instructed to use 
whatever gauge is appropriate during these "transition" 
periods (i.e. use the Nipher gauge for snow events and the 
rain gauge for rain events). 

#29 Consideration should be given to changing the orifice diameter of 
the various collectors and gauges to reflect an orifice diameter 
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equal to the diameter of the container measured across the orifice 
opening from the midpoint of the gauge or bucket wall, 
action The knife edge of the M.I.C. collector does not need any 
consideration. The storage gauge measurement is a depth 
measurement, not a volume measurement and the knife edge of 
the Nipher gauge also does not require any consideration. The 
Aerochem Metric bucket edge is less than 2 mm and is not 
considered significant. 

#30 Modificatons should be made to the M.I.C. collector to improve 

reliability, operation, and performance and to allow standardiza- 
tion of instrument performance, 

action Heaters will be installed inside the collector housing to 

maintain proper temperatures for P.C. board components and 
reduce condensation. 

#31 Modifications to Aerochem Metrics Sampler. 

action This recommendation was made in a previous audit and has 
already been acted upon. 

#32 Modification of the Metrex sequential sampler. 

action Flow rate determinations are presently more than adequate 
(calibrated by mass flowmeters). Installation of mass flow- 
meters on every sampler is too expensive. Adding time coun- 
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ters on each channel to ensure clock accuracy is considered 
over-kill. We are only concerned in ensuring that reported 
volumes are accurate, 

#33 Replace low-volume sampler pump with a more reliable pump. 

action The auditor has not demonstrated that the present pump is not 

reliable. Present pumps last, on average, over 2 years. 

Considering these pumps are running outdoors 24 hrs/day every 

day, it is felt that they are reliable. 

#34 Replace the storage gauge winter solution with an immersion heater ^ 

in water. Standardize storage gauge orifice height, 

action Storage gauges have been replaced with cumulative Nipher a 

gauges. i 

#35 Reinforce Nipher shielded gauge support stand. 

action Technologists have been instructed to do so where required. 

#36 Provide operators with caps for the Nipher shielded gauge collec- 
tion vessel. 
action These have been provided. 

#37 All sites should have GFIC's on 115 VAC supply line. 
action Technologists have been informed to ensure that all sites have 
a GFIC circuit (if not - a GFIC receptacle is Installed). 
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#38 On site receptacles should have a cover. 

action Technologists have been instructed to install covers at sites 
where problems have been encountered. 

#39 APIOS siting criteria should be added to in the following areas: 
no objects within 5 metres of a sampler; 
sites located in agricultural areas should have a windbreak 

for 360^ 

windbreaks on site are very desirable especially in the pre- 
vailing wind directions; 

due to their effect on airflow, buildings should be located if 
possible four to five heights away from the sampler; 
on site pollutant sources (e.g., woodburning stoves) should be 

avoided; 

windbreaks should be located in between roads and the 

sampler. 
action Reconmendations by the auditor such as "windbreaks on site are 
very desirable especially in the prevailing wind direction" 
are indeed desirable, however if all these were to be incor- 
porated into our criteria, the criteria would be too restric- 
tive. Present criteria are major criteria to be used for the 
APIOS network. 
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#40 APIOS should develop a formalized corrective action system. 

#41 APIOS should develop a formalized training programme for new APIOS 
network personnel and also a programme to update and upgrade exist- 
ing personnel. 

action These have been Incorporated into the QA manual. 

#42 Additional manual data checking should be carried out by technolo- 
gists and the QA Coordinator as follows: 
action Most of these recommendations have been incorporated. 

#43 Upper and lower limits of integrity checks for screening cumulative 
data are determined from historical event data. It should be veri- 
fied that these limits would not be significantly different if 
determined from cumulative historical data. 
action This verification is to be undertaken in the future. Initial- 
ly, validated event data was used since no cumulative data was 
available at that time. 

#44 Data should be screened for outliers prior to gross limit checks 

being carried out. 
action The gross limit check is an iterative procedure. The gross 
limit check is used to identify outliers. 
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#45 The groupings of sites used for the outliers tests should be 
checked to see whether any one site repeatedly experiences out- 
liers, 
action Intersite correlations are presently under review and informa- 
tion from this evaluation will be used to verify site group- 
ings. 

#46 For the outlier test, the log of parameters should be used when 

applying the Dixon Ratio Test, 
action If the distribution of the parameters shows a normal distribu- 
tion the Dixon ratio test can be used without taking the log. 

#47 Consideration should be given to reorganizing the regional techno- 
logists' responsibilities to decrease large time allocation that is 
presently necessary for travel to and from the monitoring sites. 
action This recommendation is presently being evaluated because of 
the "regionalization" of positions which has just taken 
place. 
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GUIDELINES FOR THE COMPLETION OF 
"MOE SITE DESCRIPTION QUESTIONNAIRE" 



PAGE 



1 

1 

1-2 
2 



3 
3 
3 

3 
3 

3 
3 



SECTION 

# 



II 



II 



NAME 



Site 
Identification 



Site Details 



Site Details 



ITEM 



1-8 

9 

10-14 
15 



2 
3 
4 

5 
6 

7 
8 



GUIDELINES 



Self-explanatory 

Park, governnent facility, 
private property, climat site, 
institution, conservation area 

Self-explanatory 

Modified Sangamo/MIC, Aerochen 
Metrics, Nipher, Standard rain 
gauge, Storage gauge, Lov/-vol , 
Metrex sequential, 
Fisher-Porter Rain gauge 

Precipitation chemistry, 
standard precipitation depth, 
air quality, meteorological 

Forest clearing, open area, 
field 

Flat, hill, sloping, bowl 

Self-explanatory 

Grass, grain, soil, sand, noss, 
rock, gravel 

Self-explanatory 

Walk, car, 4-wheel drive, 
snowmobile, snowshoe 

Self-explanatory 

Coniferous trees, deciduous 
trees, wires, poles, fences, 
towers. Building height above 
sampler not above grade 

Distance, direction, general 
description of area as well as 
typical heights of snow or 
amount of dust 



Cont orrl Scientillc Coiixirolion 



- AI.2 



PAGE 

# 


SECTION 

# 


NAME 


ITEM 


— ~ ■ 

GUIDELINES 


3 








Self-explanatory 


4 






11 


Land use: cultivation, orchard, 
crops, pasture, forest, water, 
livestock, garden 

Ground cover: grass, shrubs, 
gravel, soil, sand, moss, rock 


4 






12 


Describe any physical changes 
to site or surrounding area 
anticipated, i.e., new 
buildings, new industry, new 
pollution sources, new roads, 
new road surfaces, new gardens, 
new poles or wires, cut trees 
or exposed earth, etc. 


4 






13 


Locked gate, guard dog, 
flooding in spring, large snow 
drifts 


4 






14-17 


Self-explanatory 


5 
6 


III 


Site 
Influences 


1 
2 


Self-explanatory 

Access roads, paved roads, 
unpaved roads, parking lots, 
garages, major highways within 
5 kin 


6 






3 


Lakes, rivers, railroads, 
airports, canals 


7 






4 


Cultivation (specify crop 
type), orchards, pasture, 
grazing, trees 


7 






5 


Local: sewage lagoons, 
salt/sand piles, gardens, 
gravel pits, marshes, 
livestock, exposed earth, 
oil/gas wells, construction 
activity 
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PAGE 

# 



8-9 
lO-U 

12 



SECTION 



IV 



NAME 



ITEM 



13 



14 



Site Diagram 



Topographical 

Map 



Aerial 

Photograph 



Site 
Photograph 



7-9 
1 



GUIDELINES 



Area: Power plants, heavy 
industry, light industry, open 
pit mines, foundaries, mines, 
saw mills, pulp & paper plants 
within 50 km 

Towns within 5 km , important 
reference points, major urban 
centres within 50 km and 
special notes for large urban/ 
industrial centres (e.g., 
Toronto) 



Self-explanatory 



Use symbols as 
legend 



indicated in 



Attach a section of the most 
appropriate topographical map 
which shows the APIOS site and 
surrounding area 

Attach a section of the most 
appropriate aerial photograph 
which shows the APIOS site and 
surrounding area 

Include photographs from the 
four compass points with the 
sampler in the foreground 
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INFORMATION SOURCE DESCRIPTION 
FOR: MOE SITE DESCRIPTION QUESTIONNAIRE 



SECTION 



I SITE 

IDENTIFICATION 



II SITE DETAILS 



III SITE 

INFLUENCES 



QUESTION 



1. 2 

3, 4 

5 

6 

7, 8 

9 

10 

11 

12 

13 

14 

15 

1, 2. 3, 4 

5, 6. 7 

8 

9, 10 

11 

12 

13 

14 

15 

16 

17 

1. On Site 

Observation 

1. Local 
Area Sources 

2, 3, 4, 5, 
6 



SOURCE 



Operator/Regional Technician 

Auditor 

Operator/Regional Technician 

MNR/Road or District Map 

Topographical Maps 

Operator/Auditor's Observations 

Operator 

Regional Technician 

QA Manager/Regional Technician 

Operator 

Auditor/Operator 

Auditor Visual Inspection 

Auditor's Observations 

Operator 

Auditor's Observations 

Operator 

Aerial Photograph 

Operator 

Operator/Auditor's Observations 

Operator 

Operator/Auditor's Observations 

Auditor to measure with tape 

Operator 

Auditor's observation - to be 

measured with a tape 

Operator 

Operator 
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SECTION 



INFORMATION SOURCE DESCRIPTION 
FOR: MOE SITE DESCRIPTION QUESTIONNAIRE 



QUESTION 



SOURCE 



III SITE 

INFLUENCES 
(cont'd) 



8,9 



Auditor observation and 
information obtained from 
topographical map 
Operator's Observation 
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MOE SITE DESCRIPTION AUDIT QUESTIONAIRE 



I SITE IDENTIFICATION 



1. Station Name; 



2. Station Number (SIS) „ 

3. Date of Visit: ^Time: 

4. Auditor: 



5. MOE Region 

6. County: Townwship:_ 

7. Latitude ' ' _" Longitude: _ 



UTM Co-ordinates 



8. Elevation m 

9. Type of Site (park, gov't facility, private property, etc): 



10. Site Address 



Mailing Address 
(if different 
from above) 



11. Brief description of how to reach site 



12. Name of Primary Operator 

Alternate Operator _ 
Regional Technician 

13. Operator^ Address 



Mailing Address 

(if different 

from above) ____^___ 

Telephone Number Home ( ) Business j )_ 



Alternate Operator Home ( ) Business J }_ 
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MOE SITE DESCRIPTION AUDIT QUESTIONAIRE (Cont'd) 



14. Sample Types Collected at Site: 

Cumulative Wet Cumulative Dry 

Event Wet ^ Event Dry 



Other (describe)_ Special Study_ 

15. List monitoring instrumentation on site. 

Instrument Measurement 

Type/Make 
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II SITE DETAILS 



1. Physical characteristics of site (i.e. clearing, open area) 



2, Site Topography (e.g. flat, hilly, sloping): 



3. Ground slope at site:_ 

4. Ground cover at site: 



5. Access to site Summer: Good Fair Poor N/A_ 

Winter: Good Fair Poor N/A_ 



6. Method of access: Suimer: 



Winter: 



7. How close do vehicles normally approach sampler? 



8. Presence of windbreaks within lOOni of sampler: 

Type Height (m) Distance (m) Direction 



9. Where do snow and dust drift and accumulate?: 



10. Prevailing wind direction during precipitation?; 

Winter :^ 

Sunmer : ___^_ 
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II SITE DETAILS (continued) 



11, Land Use; 




•ISm 



i^tMt* 





Land 


Use 


% Ground Cover 


1. 








2. 








3. 








4. 








5. 








6. 








7. 








8. 









12. Future land use, planned or expected (construction, new 
cultivation, clearing, etc): 



13. Are there any logistical problems which prevent the sampling 
site from being easily approached or operated (locked gate, 
guard dog, flooding in spring, large snow drifts); 



14. What is the electrical power available at the site? 

(volts) (Amps) {# of circuits) 



15. Is the circuit on a: 
Receptacle GFIC 



No GFIC 



Circuit Breaker SFIC 



Battery 



16. Distance from collector receptacle to circuit panel: m 

17. What are the number of power failures: 

per month per year 



I 
I 
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III SITE INFLUENCES 

I. General potential sources of contamination/interferences 



Type 


HEIGHT(m)/ DISTANCE 
SIZE (m) 


DIRECTION 


On-Site Obstructions 








Trees 


Poles 








Wires 








Buildings 








Other - 








Local Area Sources (within 5Km) 








Sewage Lagoons 


Salt/Sand Piles 








Gardens 








Gravel Pits 








Marshes 








Livestock 








Exposed Earth 








Oil /Gas Wells 








Construction 








Human Activity 
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III SITE INFLUENCES (continued) 



2. Vehicle Related 



TYPE 


NAME 


SURFACE/ 
USAGE 


DISTANCE(m)/ 
DIRECTION 


SNOW/DUST 
CONTROL 


Highways 










Paved Roads 










Unpaved Roads 










Access Roads 
(Driveways) 










Parking Lots 










Garages 











3. Other Transportation-Related Influences 



TYPE 


NAME 


DISTANCE(km)/ 
DIRECTION 


USAGE 


COMMENTS 


Lake 










River 










Rail 










Airport 
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III SITE INFLUENCES (continued) 
4. Agricultural Activities 



TYPE 



AMOUNT 



DISTANCE(km)/ 
DIRECTION 



HERBICIDES/ 
PESTICIDES 



5. Local and Area Pollutions Sources 



F 



TYPE 



NAME 



01 STANCE (km)/ 
DIRECTION 



CAPACITY/ 
PRODUCT 



EMISSION 
TYPE/RATE 
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III SITE INFLUENCES (continued) 



6. Population Centres 



NAME 



OISTANCE(km) 



DIRECTION 



POPULATION 



7. 



Is the site representative of the topography and vegatative 
ground cover within the surrounding area? (10 - 100 km) 



8. List deviations from siting criteria and all advantages and 
disadvantages of site: 



Giixcjrcl Scienlilic Corixjr.ilion 



AI.14 - 



in SITE INFLUENCES (continued) 

9. Auditor's conments with respect to site (Is it a excellent, 
good, fair, poor unacceptable site?); 
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N 
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LEGEND OF SYMBOLS FOR USE IN SITE DRAWING 




Building 



=^= Aerial Cable 



Primary Sampler 



■> Prevailing Wind Direction 



n Low Vol 



I I I I I r Railroad tracks 



O Storage Gauge 



Poles 



A Nipher Guage 



North 



Roads 



■* Jt ** Fence 




A 



-, Brush 



Coniferous Trees 



H- Height 



g Stevenson Screen 
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Site Diagram 
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Topographical Map 




Coiuord S. JL'Mlitk C'ljqitjmtiun 



AI.21 



Site Photographs 




Cuixixxi xk-ntilk C'oiixiraiiun 



- AI.20 
Aerial Photograph 




- AI.22 - 

TYPE OF MONITORING - CUMULATIVE WET 

INSTRUMENT - MIC Collector [Formerly Sangamo Type A - Modified) 

Configuration Checks (Record results on "MIC/Sangamo Type A Data 
Sheet"F) 

1. Ensure gasket on cover contacts knife edge all the way around, 
especially on farthest position of sampler cover. 

2. Ensure sample bag fits container well. 

3. Only galvanized support should be used. 

4. Original steel U-bolts were to be replaced with stainless steel U- 
bolts. 

5. Knife edge should be level across opening. Qualitatively test by 
placing clean plastic over open wet bucket with a level on top. 
Note whether it is level, slightly out of level or not level. 

6. Check that the underside of sensor grids is not blackened or dis- 
coloured. 

7. Check that both grids are warm, not hot to the touch. 

Calibration Checks (Record results on "MIC/Sangamo Type A Data 
Sheets^^l ~ 

1. Instrument Response - Using CSC Ohm Ranger find the resistance of 
the precipitation sensor necessary to activate the sampler. A 
resistance of 220 K;^ should cause the sampler to activate. (Figure 
1). 
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2. Heating of Sensor - Measuring the voltage across the two heating 
terminals on the sensor a pulsing ACV of 5 volts should be detec- 
ted. If the voltage is not pulsing there is an instrument problem. 
Test the heater by placing voltmeter probes across heater pins 
(Figure 2). The grids should feel warm to touch not hot or cold. 

3. Clutch Adjustment - Activate and stop sampler midway by turning on 
and off switch. Place the hook of a spring gauge at the junction 
of a sampler arm and cover. Holding the other end of the gauge 
(ring) turn on sampler. The arm should start to move until the 
gauge reads 14 lbs at which point the instrument clutch should 
start to slip. Measure the clutch slip. (Figure 3). 

4. LiTTiit Switch Adjustment - Activate sampler and when sampler cover 
reaches midway turn off sampler and unplug from receptacle. Remove 
instrument screen and cover. 

i) Depress limit switch plungers (2x) until they click and then 
release. Switch plungers should spring back without sticking. 

b) Visually inspect switch plungers. Note any deformity of 
plunger. 

Plug sampler back into receptacle and turn sampler on observing 
angle of adjustment screw and limit switch plunger. 

c) Note if adjustment screw is contacting plunger on such an 
angle as to damage switch. 

d) Note if adjustment screw at end of travel is bending limit 
switch support bucket, {Figure 4). 

5. Gasket Seal on Knife Edge - Place a thin layer of clear compres- 
sable foam on the edge and allow sampler cover to move and compress 
foan will allow a check of gasket sealing ability. After a short 
interval activate sampler and inspect knife edge impression on 
foam. (Figure 5). 

6. Sampler Delay Time - Short sensor with metal strip or moistened 
finger and start stopwatch. Stop stopwatch when sample cover 
starts to return to wet side of instrument. 
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FIGURE 4 
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MIC/MODIFIED SANGAMO TYPE A DATA SHEET 



Site: 



Serial #:_ 
On Stand? 



Model #: 



Height above ground 



(Bucket) SAMPLE ORIFICE (cm) 



(Bag) SAMPLE ORIFICE (cm) 



Level 



WET 



WET 



Precipitation Sensor Checks 



Knife edge gap 

Instrument Response 

Heating of Sensor 

Warm to touch 



mm Delay Time 



kfl Clutch Adjustment 



volts pulsing? 



sec 

lbs 



Blackened/Discolored? 



Comment on: 



Limit Switch Adjustment: 



Gasket Seal on Knife Edge: 



Hood Movement; 



Bag Fit: 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC 



OPERATOR: 
SITE: 



Operator's Duties 



1. 



2. 



What time does operator change sample bags? If he. 
can't change at the proper time, what does he do? 
{phone technican, change bag when he can, contact 
alternate operator). _^ ^ 



Yes 



Does the operator, while removing the sample bag from 
the sample bucket, allow for any chance of 
contamination of the precipitation sample? Several 
points that may present contamination are: 

is sample bucket removed from sampler prior to 

changing sample bag? 

does the operator wear clean, new gloves to 

remove the knife edge? 

does he handle onlx the outside of bucket and 

knife edge? 

is he careful not to contaminate knife edge when 

its taken off? 

is bag removed in such a way as to not introduce 

contamination from top and outside portions of 

bag? 

is operator careful not to stand directly over 

pail? 

is there a potential for contamination by hair, 

coat, etc.? 



Comments: 



No 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 

3. Are the sample bags properly handled to prevent 

damage, sample loss and allow for proper handling by 
laboratory staff? Several points should be checked: 
is air forced out of bag before sealing?. 
is bag tied near enough to the top so that it 
can be leached for metals analysis, but far 
enough down so that none of the contaminated 
surfaces are exposed to sample? How is bag tied 
and sealed? 



Comments : 



does operator use a container to store and 
transport sample? 



are bags promptly labelled {What information is 
recorded on label)? 



for winter operation with long tube how is bag 
removed? ^^_^_ 



where does operator store sample for pick up by 
technician? (winter and summer). 



how long does it take the technician to come and 
pick up samples? 



Yes 



No 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 

4. Are the sampler components carefully cleaned? 
Several points to be checked: 

does operator clean the following before put- 
ting in new bag: 
Sensor grids with H2O and ETCH? 
Hood gasket with H2O and ETOH? 
does he use new gloves and Kimwipes for above? 



what is the cleaning procedure under cold 
conditions? _^ 



Comments: 



S. What problems does the operator experience with the 
sampler? How does he handle them? Some things that 
can happen; 

does wet sample bag get stuck in container when 
being removed? _^__ 



Yes 



what does operator do if this happens? 



what procedure is followed if the bag comes out 
damaged? 



No 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 



how often are leaks observed in bags? 



what does he think causes it? 



Comnients: 



In placing a new sample bag in the collector, does 
the operator observe the following points: 

are new gloves worn when touching inside of 

bag? 

is operator careful not to touch inside of bag 

any more than is necessary? 

is bag forced to bottom of container by filling 

with air? 

is operator careful not to put hand inside bag 

when installing it? 

is enough bag left over the edge to permit knife 

edge to hold bag? 

are new gloves used to flatten bag against the 

pail? 

is collar cleaned, rinsed and dried before 

replacing on container? 



Yes 



No 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 



Does the end result produce a well fitting bag? 
(smooth surface, no overhang). 



Comments; 



7. Does the operator have a good knowledge of what he is 
expected to do? Specific points that the operator 
should be asked: 

does operator have written instructions for 

procedures? 

which procedures does he have? 

does he think he was adequately trained? 



when was the last time the regional technician 
reviewed procedures with him? 



does he have any particular problems with the 
sampling? 



Yes 



No 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 



Yes 



Conments: 



8. Does the operator have sufficient supplies on site? 
He should be asked: 

does he receive regular shipment of supplies or 
as needed? 



No 



does he have enough of the following: 

sample forms? 

bags? 

DIH2O and ethanol? 

kimwipes? 

gloves? 

has he ever run out of supplies? 



Comments: 



9. What comments does the operator have about the 
programme? 

does the operator understand why he is 
collecting the samples? 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/HIC (continued) 



Operator's Duties 



Yes 



does operator understand why all the special 
precautions (glove, cleaning bags etc.) are 
required? ^ : — 



does operator get any feedback on programme? 
Would he like feedback? How often? 



No 



does he have any special comments with regards 
to the sampling, site, technician or programme 
as a whole? 



does he feel the pay is adequate? 



Comments: 



10. Does the operator check sampler performance on a 
regular basis? 

does he check that hood seals tightly 
does he check that the sensor grid is warm? 
what other instrument checks does he perform 
periodically? 



how often does he check the instrument? 
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CUMULATIVE PRECIPITATIOM AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 



Comments: 



11. Does the operator maintain a site log book? 
what type of information is recorded _ 



how often is information recorded? 



Yes 



Comments: 



12. Are sample submission sheets correctly and completely 
filled out? _^__ 



13. Does the Regional Technician visit the site? 

how often? 



does he talk to the operator on each visit, or 
is the sample just left at a location for 
pickup? _^ 



No 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 



do the technician and operator discuss sample 

problems? 

other than talk to the operator what else does 

the technician do on his site visits? 



Comments: 



Yes 


No 
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TYPE OF MONITORING - CUMULATIVE WET 

INSTRUMENT - STORAGE GAUGE 

Configuration Checks (Record results on "Storage Gauge Data 
Sheet") 

1. Storage gauge opening should be level. Qualitatively test by 
placing level across opening. Note whether it is level, slightly 
out of level or not level. 

2. Measure the height of the bottom of the storage gauge from the 
ground. 

3. Measure the height of the orifice of the storage gauge from the 
ground. 

4. Visually note any floating debris in guage or adhered to inside 
gauge walls. Using metre stick check for any sunken debris. 



Calibration Checks (Record results on "Storage Gauge Data Sheet") 

1. Gauge Opening - Measure inside opening of storage gauge with a 
ruler. Repeat at 90* from first reading. 

2. Solution and Capping Layer - Visually note whether solution is 
methanol /ethylene glycol (slight alcohol - like odor) or water (no 
odor). 
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STORAGE SAUGE DATA SHEET 



Site: 



Serial #: Model #: 



On Stand? Storage Gauge Height above ground_ 

Orifice Height above ground 



How is the base of storage gauge supported? 



Sample Orifice (cm) N-S level N-S 

E-W E-W 



Solution Type: Methanol Ethylene Glycol 
Water 
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CUMULATIVE PRECIPITATION AUDIT - STORAGE GAUGE 



OPERATOR: 

SITE: 



Operator's Duties 



1. Does operator take a storage gauge measurement? 



2. How does the operator take the depth measurement? 



3. Does the operator have a thermometer present for 
winter time measurements? 



Comments: 



Yes 


No 
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T,PE OF mmmi - otulative dry 

IHSTRO«ENT - «TR« LO«-Va SADLER 



Filter Pack Plate 




ironmental 
|ing 



ground level 



. . of the s«li"5 eonn^u^Uon. (see F^-e 1), 



1 . Note a"J «-' - X «+> 

i «« ualls of rota 
KT ack ring on waiis "' 

,. Observe fl^ ^^^ ^ "clness. colour ^dar. .UC .re., 
nieter? If yes describe ring 

brown)' 

n utatively check by placing 

3. - -"^n^rr^re:^orrr n . ... ..-. 

level across filter pa 
out of level or not level. 



1. 



^'^^''''''''^^''^^ ^ nff Lo-Vol Sampler and dis- 

,/mln mass flowmeter ,n U.e. Turn s P ^^^^^ ^_^ ^^^^^_ ,,, 

1 .Uow 30 "-'«;°;X ;„,r.e connected to a -i^Ha, 
output of the mass "■«•"«",; ''°^ „„ „ntrol «)just on the 

cover . ranae of 0-5 l/m1n. 
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2. Volume - Calibration of Volumetric Counter (Gas meter) The instru- 
ment set up for calibration is the same as in the "Calibration of 
Rotometer" except that once the flow controller has been set to 
approx. 2 1/min it should not be adjusted. When unit on volumetric 
counter changes start stop watch and record volumetric counter 
reading and mass flowmeter reading. Each time the volumetric 
counter changes (approximately 5 minutes) record volumetric counter 
reading, mass flowmeter reading in VDC, Low-Vol rotameter reading, 
and time (record results in "Volume Counter Calibration Table"). 
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METREX LOW-VOL SAMPLER DATA SHEET 



Site: 



Serial #: _^ Model #: 



On Stand? Height above ground_ 



Sample Orifice (cm) N-S "level N-S 

E-W E-W 



Sampling line and tower in good condition? 
Kinks in suction line? . 



Tower firmly supported, straight and steady? 
Low-Vol sampler - covered? __^ 



located so as not to be influenced by other nearby 
samplers or objects? _^ 



Rotameter Condition - grey, brown or black ring? 
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Rntameter Calibration 


Temperature 


















Reading # 


Low-Vol 

Rotameter 

Reading 

1/min 


Voltmeter 

Reading 

(VDC) 


Mass Flowmeter 
Reading 
(1/min) 






1 


, 










2 












3 












4 












5 












6 












7 












8 












9 












10 
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Reading # 


Time 
(min & sec. } 
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Reading 
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Reading 
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Low-Vol 
Rotometer 
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1 
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CUMULATIVE DRY DEPOSITION - LOW VOL 



OPERATOR: 

SITE: 



Operator's Duties 

1. Describe the procedure used by the operator in 
handling filter packs. Does he do the following? 
handle filter packs only by using a plastic bag 
or on the filter pack? ^__ 



where are the filter packs stored? 



Comments: 



Yes 



No 



What information is recorded on data sheets? 
flow rate - start and finish? 
volumetric reading? 
time of filter pack changeover? 
operator comments? 



Comments: 



3. What training did operator receive on the sampler 
handling filter packs? 



has he been retrained since? 

does he feel training was adequate? 



Comments: 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER 



OPERATOR: 
SITE: 



Technician/Operator's Duties 

1. Describe operators procedures prior to lowering the 
boom? Does he do the following? 

field sheet prepared for each filter pack or 

tower? 

are sampling channel flows transferred to data 

sheet? 

is the time showing on mechanical clock correct? 

is the flow rate checked? 

1s the sampler turned off before the boom is 

lowered? 



Comments: 



2. After the boom has been lowered, describe the pro- 
cedure used to reload filter pack. Is the following 

done: 

is a new clean sample bag used to remove filter 

pack? 

what type of bag is used? 



are either the filter packs or bags labelled 
corresponding to sample channel? If not how are 
the filter packs identified? 



is the filter pack sampling for that day 
transferred to another sampling channel? What 
channel? . 



Yes 



No 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER (CONT'D] 



Technician/Operator's Duties 

Yes No 

is the sampler turned off while filter packs are 

being removed? 



Comments: 



3. 



Yes 



Describe procedure used to reload filter packs. Is 

the following done: 

is the filter pack installed on tower while 
still in bag or is it removed by hand and then 
installed? ^ 



when are the sampling channels zeroed?^ 



is the sampler restarted when the boom is on the 
groLind or after being raised?__ 



does the technician cycle through all channels 
after sampler is restarted? 



4. What information is recorded on the sample sheets and 
in the site log book? 



How frequently is the sampler checked, daily, weekly? 
Who does the checking the technician or someone else? 



No 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER (CONT'D) 



Technician/Operator's Duties 

6. At the laboratory how long do the filter packs usual- 
ly sit before being unloaded?^ . 



Yes 



8. 



Does the technician unload filter packs in a: 
laminar flow chamber? 
fume hood? 
office? 
laboratory? 

Are there any exposed contaminants in filter proces- 
sing area? 

anyone smoking nearby ?_ 



No 



any exposed chemicals - especially HgSO^ or HNO3 



is the area generally clean or dirty?_ 



Comments: 



9. What kind of work surface does the technician use? 

teflon coated table top? 

lab bench? 

lab bench with plastic on top? 

office desk? 
Conments: ^ 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER (CONT'D) 



Technican/Operator 's Duties 

10. Is the technician's system for labelling bags easily 
understandable and logical? (describe the system 

used) 

is it done all at once prior to unloading any 
filter pack?^ 



is it done as filter packs are unloaded?_ 



Comments: 



11. Describe the procedure used for handling the 
filters? Are the following points observed? 

are teflon tweezers used for filter handling? 

are tweezers stored in methanol prior to use? 

are they wiped with new Kimwipe before touching 

filters? 

are separate tweezers used for impregnated and 

unimpregnated filters? 

are Nylon and Impregnated W41 filters folded 

twice before placement in sample bags? 

are gloves worn during filter pack unloading? 



Comments: 



12, Describe the procedure used for the washing and 
drying of filters? Is the following done: 

where are filter samples stored after unloading 
from filter packs? 



Yes 



No 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER ^CONT'D) 



Technican/Operator's Duties 



are filter packs completely disassembled prior 
to washing? 

are the filter packs rinsed after washing? What 
rinses are used? 



how many of each type of rinse are performed?_ 



Yes 



No 



how are the filter packs dried? If air dried, 
are they covered during drying? What covering 
is used? 



if dried in an oven, how long, what oven 
temperature? , 



are the filter packs used after drying? If not, 
how are they stored? . 



Comments: 



13, Describe the procedure used for reloading of filter 
packs. Is the following done: 

does the technician clean off the area where 
filter packs will be loaded? 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER (CONT'D) 



Techm'can/Operator's Duties 



is the work surface a lab bench, Teflon overlay 
board, "Kimwipes" over lab bench or other area? 



are cleaned tweezers kept in separate breakers 
one for un impregnated and one for impregnated 

filters? 

when removing filters from bag does technician 

push his hand into bag or only tweezers? 



when placing collar on filter pack is technician 

careful not to get hand near open exposed face 

of filter? 

is the inlet of the filter pack checked to 

ensure the 0-ring has not failed? 

is the loaded filter pack numbered and stored in 

a new "Zip-lock" or "Whirl-pak" bag? 



how are loaded and bagged filter packs stored 
prior to shipment to field?_ 



Comments: 



14. Are any of the filters from Toronto ever damaged? 
(i.e. nicks out of the sides). If so what is done 
with them? ^ 



Yes 



No 



Concord Sticntitic Cor]X)wli<)n 



- AI.50 - 

TYPE OF MONITORING - EVENT WET 

INSTRUMENT - AEROCHEM METRIC 

Configuration Checks (Record results on "Aerochem Metric Data 
Sheet") 

1. The Aerochem Metric has a reputation for being blown over in winds. 
The sampler should be securily mounted by either stakes, wire or 
cross pieces. Insure that the sampler can not be pushed over. 
(Figure 1). 

2. The instrument should be level. Qualitatively check by placing 
clean plastic over the bucket with a level on top. Note whether it 
is level, slightly out of level or not level. 

3. Check cleanliness of a) gasket, b) sensor, c) sensor body. 

Calibration Checks (Record results on "Aerochem Metric Data 
Sheet") 

1. Heating of Sensor - When sampler is not activated sensor should be 
cold to touch. When activated, the sensor should get hot after 
approximately 5 minute delay, and stay hot until activating precip- 
itation evaporates. 

2. Cover Seal Integrity - A thorough visual inspection of cover check- 
ing for: a) holes in polyethylene gasket cover; 

b) signs of gasket foam crumbling; 

c) moisture on inside surface of polyethylene. 
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FIGURE 1 
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AEROCHEM METRIC DATA SHEET 



Site: 



Serial #: 



Model #; 



Sample Orifice (m) N-S 

E-W 



level N-S 
E-W 



Secured by: stakes 
wire 



cross pieces 



sturdy? 



Cleanl iness - Gasket 

- Sensor 

- Sensor Body 
Any holes in gasket cover? 
Is gasket foam crumbling? 

Any moisture on inside surface of cover 
Sensor heating properly? 



Yes 


No 



































Comments: 
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EVENT PRECIPITATION AUDIT - AEROCHEM METRIC 



OPERATOR: 
SITE: 



Operator's Duties 

1. What time does the operator check the sample bag for 
precipitation? If he cannot change at the proper 
time, what does he do? (phone technician, change bag 
when he can, contact alternate operator). ^ — 



Yes 



Does the operator, while removing the sample bag 
from the sample bucket, allow for any chance of 
contamination of the precipitation sample? Several 
points that could lead to contamination are: 

is the sample bucket removed from the sampler 

prior to changing sample bag? 

does the operator wear clean (newl plastic 

gloves to handle the bag? 

is the outside of the bag folded up allowing 

contamination to enter bucket? 

is the operator careful to not stand directly 

over sample bucket while removing sample bag? 

Is there potential for contamination from 

operator's hair, clothes etc.? 



Comments; 



No 
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EVENT PRECIPITATION AUDIT - AEROCHEM METRIC (continued) 



Operator's Duties 

3. After the bag is removed from the sample container, 
1s air evacuated from the bag after being 
removed from container? 
is the top of the sample bag tied? 
how far from the top of bag is it tied? 
what is the sample bag tied with? 
any chance for othe contamination (clothing, 

etc)? 



Yes 



Mo 



Comments: 



4, For reloading the sample bucket: 

in placing new bag in sample container, does 
operator wear gloves? 

does the operator touch the inside of bag when 
loading into sample container with only new, 

clean gloves? 

any chance for other contamination (clothing, 

etc.)? 



Comments: 



5. When the new bag is loaded in the sample bucket: 

does the bag fit the container well? 

how much of a decrease in orifice size results 
from poor bag fit? How much is the sample area 
reduced (^)? , 



Comments: 
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EVENT PRECIPITATION AUDIT - AEROCHEM METRIC (continued) 



Operator's Duties 

6. Describe how the operator transfers the precipitation 
sample from the sample bag to the sample bottle? 
does the operator inspect sample bottle for 
flaws? 

does he swirl bag and contents around prior to 
decanting? 

how is sample melted if partially or completely 
frozen? 



does operator pour sample directly from bag into 

sample bottle? 

does he use a clean funnel? 

does he elevate one bottom corner, clean, and 

cut it? 

describe cleaning procedure of bag and scissors? 



if sample volume exceeds bottle contents, is 
excess poured into graduate and measured? 
where is the sample transfer area? 



is a clean sample preperation area used? 



Comments: 



Yes 



No 
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EVENT PRECIPITATION AUDIT - AEROCHEM METRIC (continued) 



Operator's Duties 

7. Sample bottles should be labelled to correspond with 
date submission sheets: 

what information is placed on sample label? 



is label placed on cap or bottle? 



where are samples stored for pickup by 
technician? _^_ 



how long are samples usually stored prior to 
pick-up by technician? 



Comments: 



Yes 



No 



8. What information is entered on the data sheets? 

is any information not filled out or incorrectly 
filled out? 



9. Is there a log book on site? 

what information is recorded? 



how often is information entered into log book? 
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EVENT PRECIPITATION AUDIT - AEROCHEM METRIC (continued) 

Yes 



Operator's Duties 

10. When was the last time technician reviewed sampling 

procedures with operator? . 

how often are procedures reviewed? 



NO 



does the operator think procedures should be 
reviewed more often? ^ 



does he think he was adequately trained 

does he have any particular problems with the 

sampling 



Conmients: 



U, 



Does the operator have sufficient supplies to last 
for one week, one month, six months? Has the 
operator ever run out of supplies? ^ 



12. 



What comments does the operator have about the 

programme? 

_ does the operator undertand why he is collecting 

the sample? 



does he understand why all the special 
precautions (gloves, cleaning, etc.) are 

required? 

does operator get any feedback on programme? 

Would he like some? How often? 



does he feel the pay is adequate? 
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TYPE OF MONITORING - EVENT WET 

INSTRUMENT - STANDARD RAIN GAUGE 

CONFIGURATION CHECKS 

(Record results on "Standard Rain Gauge" Data Sheet) 

1. Measure from collector rim to ground. 

2. Measure from top of grass to collector rim. 

3. Inspect funnel and graduate for cracks and cleanliness. 

4. Determine height of and distance to nearest obstruction, including 
sampler. 

5. Insure that gauge is emptied each morning. 

6. The instrument should be level. Qualitative check by placing level 
over gauge orifice. Note whether it is level, slightly out of 
level or not level. 

7. Pour pre-measured 50 ml aliquot into gauge and record reading. 
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STANDARD RAIN GAUGE DATA SHEET 



Site: 



Serial # 



Model # 



Collector rim to ground 
Collector rim to grass 

Graduate (Yes, No) 
Funnel (Yes, No) 



Cracked 



Clean 



Obstruction 





Height 

of Obstruction 

(m) 


Distance 

from Gauge 

(ni) 









Gauge Level 



Gauge Reading 



(mm) 
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EVENT PRECIPITATION - STANDARD RAIN GAUGE 



OPERATOR: 
SITE: 



Operator's Duties 

1. Is the standard gauge checked each day? 

2. How does the operator handle the following 
situation: 

if water level (meniscus) falls between two 
graduated marks? 



what is recorded if level is below 0.2 on scale? 



if there is water that has overflowed the 
graduated cylinder, what does the operator do? 



what is done for freezing rain? 



Yes 


No 
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SITE AUDIT SUMMARIES 
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TABLE 3.1 



Sites Visited During the Phase II Site Audits 



Site 



Type 



Merlin 

Port Stanley 

Pickle Lake 

Forbes Township 

Quetico Centre 

Ear Fall s 

Experimental Lakes Area 

Dorset 

Whitney 
Coldwater 
Gowganda 
Winisk 
Whitman Creek 



Cumulative Wet 

Cumulative Wet & Dry 

Cumulative Wet & Dry 

Event Wet 

Cumulative Wet and Event Wet 

Cumulative Wet & Dry 

Cumulative Wet 

Cumulative Wet & Dry, Event Wet 

& Dry 

Cumulative Wet 

Cumulative Wet 

Cumulative Wet & Dry 

Cumulative Wet 

Event Wet 
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MERLIN 

Site Evaluation 

Advantaoes: 

representative of topography and qround cover 

no on-site obstructions 
representative of area 
good access year round 
no major industrial sources 

Disadvantages: 

incomplete windbreak 
adjacent to a barn 

Instrument Evaluation 

Sangamo Type A 

low resistance on sensor 
instrument in good working order 

Storage Gauae 

orifice slightly out of round 
slightly off level 
well supported 

Corrective Action Required 



none 
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PORT STANLEY 

Site Evaluation 

Advantages: 

representative of topography and ground cover 
no on-site obstructions 
representative of area 
good access year round 
no urban influences 

Disadvantages: 

incomplete windbreak 
proximity to treatment plant 

Instrument Evaluation 

Pr-fmary MIC Type A 

low resistnace on sensor 
good working order otherv;ise 

Secondary MIC Type A 

low resistance on sensor 
hood movement not smooth 
good working order otherwise 

Primary Storage Gauge 

well supported 

level 

gauge orifice out of round 

Secondary Storage Gauge 
well supported 
level 
gauge orifice out of round 
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Primary Low Volume 

1 guy wire on tower broken 
good working order 
good shelter 
wen supported 

Secondary Low Volume 

good working order 
good shel ter 
well supported 

Operator Evaluation 

Operator: John Apfelbeck 

uses only 1 pair of gloves to install and remove bags 
operator requires upgrading on sample handling techniques 
good attitude, very conscientious 

Corrective Action Required 

retrain operator 
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PICKLE LAKE 

Site Evaluation 

Advantages: 

representative of ground cover and topography 

representative of area 

co-located at Climat and CAPMoN sites 

no local or area sources 

no agricultural or large urban influences 

minimal vehicular influences 

Disadvantages: 

on site obstructions - trees 20m E/SE 

- tower 21m NE 
no windbreak to the west towards the lake 

Instrument Evaluation 

MIC 

one grid blackened 

clutch adjustment too strong 

overal 1 performance good 

Storage Gauge 

well supported on wooded stand 
level 

Low Volume 

good working order 

supported on a good stand and in a wooden enclosure 
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Operator Evaluation 

Temporary Operator: Ron Marsil 

follows outlined sample handling procedures 
checks sampler daily 

Corrective Action Required 

remove trees 

perform minor maintenance on MIC 
train new operator 
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FORBES TOWNSHIP 

Site Evaluation 

Advantages: 

representative of topography and ground cover 

no on-site obstructions 

well removed from urban and agricultural influences 

good access year round 

representative of local area 

complete windbreak 

Disadvantages: 

proximity to power plant 

Instrument Evaluation 

Aerochem 
Primary 

good working condition 

dry bucket in place, possible source of contamination 

Secondary 

good working condition 

dry bucket in place, possible source of contamination 

Standard Rain Gauge 

Primary & Secondary - gauges level, clean and no cracks in 
graduate 
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Operator Evaluation 

Operator: Mary Ingles 

follows all sample handling procedures outlined in APIOS 

Technical and Operating Manual 

excellent attitude 

very conscientious in sample handling 

Corrective Action 

remove dry buckets from Aerochems 
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QUETICO CENTER 

Site Evaluation 

Advantages: 

no on-site obstructions 

good ground cover 

reliable power source 

easily accessible 

windbreak complete except to S-SE 

representative topography and ground cover 

site representative of area 

Disadvantages: 

proximity to road 

Instrument Evaluation 

MIC - Sangamo 

one sensor blackened, hot to touch 

instrument supported well and on a good stand 

sampler activated with 200 n resistance 

sampler generally in good working order except for 

blackened sensor grid and low resistance on sensor 

Aerochem 

sampler in good working order 

water dripping from sensor head 

dry bucket in place, possible source of contamination 

poor seal on bucket 

new motor and sensor installed 

Standard Rain Gauge 

gauge level, clean and no cracks in graduate 

Qjncorcl Scienlilk Cdqxjralion 



All. 10 



Operator Evaluation 
Operator: Austin Oonohue 

operator leans over sanple bag when changing bag 

air not forced out of bag before sealing 

gloves not worn to clean gasket, hand touches gasket 

same gloves used to handle outside of bucket and install 

new bag 

the instrument body and component parts should be cleaned 

more often 

operator has a good attitude and would be an excellent 

operator if retrained 

Corrective Action Required 

sensor grids on MIC to be changed 

dry bucket should be removed from Aerochem 

Aerochem should be adjusted to provide a good seal on wet 

bucket 

operator to be retained and periodically checked to ensure 

that he is following proper sample handling procedures 
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EAR FALLS 

Site Evaluation 

Advantages: 

representative of topography 
good accessibility and security 

Disadvantages: 

poor ground cover - gravel 

on-site obstructions 2 trees 10 m NE {- 8 m tall) 

gasoline tank 8 m NE 

proximity to parking lot or roads 

adjacent to storage area and construction (garage) 

ground slopes E-SE 

light industry 3 km W 

Instrument Evaluation 

MIC 

limit switch needs to be replaced and set properly 

chain needs to be tightened 

no other problems with instrument 

supported well and level 

Storage Gauge 

supported on a wooden stand 
level 

Low Volume 

not operating at time of audit 

well supported inside a housing unit 
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Operator Evaluation 

Operator: Harold Atkinson 

operator not present at time of audit 

Corrective Action Required 

site has many deviations, some of which can be ameliorated 

(e.g. cut trees, plant grass, clean-up area, etc.) 

deviations which cannot be improved include proximity to 

open ore pit mine, vehicular influence, light industries 

and poor ground slope 

relocate site, if not possible, then historical data 

should be reviewed to determine if valid samples are being 

collected. Site should be removed from the APIOS network 

if data indicates that invalid samples are being 

collected 

perform maintenance on MIC e.g. replace limit switch, 

tighten chain 

operator should check instruments frequently (twice per 

week) while construction is on- going 

low vol down for minimum of 5 days prior to audit; 

instrument had been unplug by contractor 
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EXPERIMENTAL LAKES AREA (E.L.A.) 

Site Evaluation 

Advantages: 

representative of topography and ground cover 

ground cover: moss, rock and blueberry plants 

well removed from urban and agricultural infuences (remote 

site) 

representative of local area 

located at a research facility 

very little vehicular traffic in the area 

Disadvantages: 

on-site obstruction, tov;er 14 m HE 

incomplete windbreak - forest fire destroyed trees in 

surrounding area 

Instrument Evaluation 

MIC 

clutch adjustment too strong > 30 lbs 
motor humming when not operating 
overall performance very good 
supported properly and level 

Storage Gauge 

well supported on wooden stand 

level 
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Operator Evaluation 

Operator: Ken Beatty 

operator feels sample handling area too cramped resulting 

in poor sample handling technique 

operator would like more information on APIOS as well as 

complete Technical and Operating Manual 

conscientious - checks sampler daily 

follows sample handling procedures 

Corrective Action Required 

move sampler away from tower 

perform minor maintenance on MIC - loosen clutch pad 
technologist to forward complete APIOS Technical and 
Operating Manual to operator 
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DORSET 

Site Evaluation 

Advantages: 

located at government research facility 

representative of topography and ground cover 

operated by APIOS regional technologist 

good accessibility 

co-located at CAPMoN, CLIMAT sites 

no on-site obstructions 

no industrial or urban influences 

Disadvantages: 

proximity to gravel & sand road parking lot, however, a 
bank of trees separates the road from the sampler 

Instrument Evaluation 

Primary Sangatno 

clutch adjustment too strong > 35 lbs 
instrument in good working order 
well maintained 

Secondary Sangamo 

clutch adjustment too strong > 35 lbs 

limit switch requires adjustment - screw not making proper 

contact with switch 

motor runs or after instrument is turned off 

"C" clamp needs adjustment 

on/off switch not working properly 

hood drops suddenly 

good seal on knife edge 
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Primary Low Vol 

. stand well supported and level 

*.v,^^inn - Should be changed 
. sample line weathering snou 

. low vol in good housing 

. operating properly 

. slightly off calibration 

Secondary Low Vol 

_ stand well supported and level 

low vol in good housing 
. operating properly 

Primary Storage Gauge 

- orifice out of round 
well supported on stand 
not level 
"U" bolt not tight 

Secondary Storage Gauge 

. orifice out of round 
well supported on stand 
not level 
"U" bolt not tight 



operator: Cumulative MetworR - OP. V a rto 
Event Network - G. Carlyie 



Cumulative 

. regional technologist 

very good attitude 
secondary MIC requires maintenance 
: event operator needs refresher on sample handling 
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very little chance of sample contamlnat-ion when sample 
changed 

Corrective Action Required 

event operator requires retraining 

- secondary MIC should be serviced 

- consideration should be given to paving parking lot and 
access road 
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WHITNEY 

Site Evaluation 

Advantages: 

representative of topography 

remote 

no agricultural or urban influences 

no vehicular activity in area 

no on-site obstruction 

accessible year round 

acceptable windbreak 

Disadvantages: 

proximity to sawmill 
proximity to swamp 
exposed earth 

Instrument Evaluation 

MIC 

clutch adjustment too strong 

limit switch needs to be adjusted - too much pressure on 

bracket 

overall performance good 

Storage Gauge 

well supported on wooden stand 

level 

mesh screen installed to keep birds out 
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Operator Evaluation 

Operator: Rick Hawkins 

operator not present at time of audit 

Corrective Action Required 

seed or sod to cover exposed earth 
perform regular maintenance on MIC 
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COLDWATER 

Site Evaluation 

Advantages: 

representative of topography and ground cover 

no vehicular influences 

complete windbreak 

no on-site obstructions 

accessible year round 

Disadvantages: 

open pit limestone quarry 10 km E (blasting 2-3 times per 

week) 

Instrument Evaluation 

Sangamo Type A 

very good working order, no problems 

Storage Gauge 

well supported on wooden stand 
gauge orifice out of round 

Operator Evaluation 

Operator: Ron or Sharon Hancock 

operator does not follow sampling protocol 

folds over bag before it is tied 

does not force air out of bag before sealing 

does not always wear gloves when changing samples during 

winter months 
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sample bags not labelled 
does not wear gloves to clean gasket 
touches inside of bag with ungloved hands 
operators have an excellent attitude and are very enthus- 
iastic 

Corrective Action Required 

operators should be retrained and then checked on period- 
ically (every six months) 

historical data should be reviewed to see if the limestone 
quarry affects the integrity of the samples collected 
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GOWGANDA 

Site Evaluation 

Advantages: 

well removed froni any major sources of pollution 

well removed from any agricultural or urban influences 

representative of local topography and ground cover 

representative of area 
no ground slope 

Disadvantages: 

poor access during winter months 
tower (20 m SE) on-site obstruction 
poor windbreak 

MIC 

clutch set too strong > 35 lbs 

overall performance, good 

sample bucket slightly out of round 

Storage Gauge 

supported by a wooden stand 
contained transmission oil and water 
level 

Low Volume Sampler 

flow calibration not performed due to problems with the 

mass flow meter 

overall performance of low vol, good 

located in good shelter well supported 
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Operator Evaluation 

Operator: Bonnie Trajkowicz 

follows sample handling procedures outlined in APIOS 

Technical and Operating Manual 

operator feels she was not adeauately trained 

over 1 year since procedures reviewed with operator 

operator should be retrained - even though she is follov;- 

ing the outlined procedures, she is not confident about 

the quality of the samples she collects 

operator has an excellent attitude demonstrated by her 

concern for her sample handling technique 

Corrective Action Required 

retrain operator 

minor maintenance should be performed on MIC 
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WINISK 

Site Evaluation 

Advantages: 

representative of topography and ground cover 

remote 

representative of area 

good access year round 

no on-site obstructions 

Disadvantages: 

snowmobile road adjacent to sampler 

sampler located at airport - possible contamination from 

airplane exhaust and human activity 

diesel generator 24 hrs/day 

Instrument Evaluation 

Sangamo Type A 

not operational when auditor arrived 
sensor grid broken 
clutch set too strong > 35 lbs 
possibly not functioning for months 

Storage Gauge 

storage gauge empty at time of audit 

no storage gauge results reported during months preceeding 

audit 
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Operator Evaluation 

Operator: Joe Bird 

does not change samples at proper time 
not careful with knife edge so as not to contaminate it 
sample bags never labelled 

operator performs no instrument checks, cleaning or main- 
tenance 

does not wear new gloves to pat down bag during 
installation 

operator claims he was never trained 

operator claims he is not being paid and therefore has 
little interest 

operator has a very poor attitude 
submission sheets not filled out properly 
recommend a new operator be found and properly trained 
since it is doubtful that present operator can be re- 
trained to operate the site properly 

Corrective Action Required 

move site away from snowmobile road or have snowmobilers 

use a different route 

replace operator or remove site 
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WHITMAN CREEK 

Site Evaluation 

Advantages: 

representative of topography and ground cover 

good access year round 

well removed from pollution sources and urban influences 

no on-site obstructions 

representative of local area 

Disadvantages: 

unreliable power source, numerous events missed during 

winter months 
proximity to road 

Instrument Evaluation 

Aerochem 

sensor sensitivity requires slight adjustment, sampler 
does not open for short duration (<5 minutes) events 
instrument in good working order 
better bag fit due to holes in sample bucket 

Standard Rain Gauge 

gauge level and has no cracks in it 
graduate requires cleaning 
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Operator Evaluation 

Operator: Sophie Angement 

follows all sample handling procedures outlined in APIOS 

Technical and Operating Manual 

excellent attitude and always informs technician of 

problems 

Corrective Action Required 

if a reliable power service cannot be obtained at this 
location the site should be moved to a location where 
reliable power is available, too many events missed 
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rHOH THE QUALITY CONTROL HANDBOOK FOR Revised Harch 13, 1980 

PILOT WATtRSHEO STUDIES, PLUARu, IJC J. L. Clark 



RE PORTING LOW LEVEL DATA 

There arc specific problems In the reporting of low level data which are 
issociated with the question: Is a substance present? While this question is 
seldoffl germane In IJC work concerned with loading estimation. It has so 
Influenced thinking about reporting low level data that It seems best to 
consider It In some depth before dealing with how such data are to be reported 
for IJC purposes. 

In »iswer1ng the question "is a substance present?" there are two 
possible correct conclusions which nay be reached. One may conclude that the 
substance Is present when U is present, and ore may conclude that the 
substance Is not present* when It Is not present. Ccrversely, there are two 
possible erroneous conclusions which may be reached. Oie may conclude that 
the substance Is present when It- is not, and one may conclude that the 
substance Is not present when It Is. The first kind of error, finding 
»0(neth1ng which Is not there, is called a TYPE I ERROR. The second kind of 
error, not finding something which is there, is called a TYPE II ERROR. 

These two types of errors are Illustrated in the material that follows, 
using the result which might be obtained from a single analysis when the 
substance Is not present to illustrate Type I error and the Inferences that 
Bight be drawn from a single analysis at two different actual concentrations 
to Illustrate Type II error. 

Of course inferences as to water quality are seldom, if ever, based on the 
r«suU of a single analysis. A single result Is used here to simplify the 
txposltion. 

*Since Avogatiro's number Is very large, a pedant could argue th«t one should 
never claim that a substance is not present. A conrion sense meaning of not 
present is intended here. i.e. if measurement Is being made In micrograms per 
litre the presence of a few nanograms per litre Is Irrelevant. 
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If the standard deviation, o, of v\ riftlytlcal procedure has been 

determined »t low C(^centrat1ons Including 0, then the probability of making a 

Type I error can be $et by choosing an ipproprUte a level to determine the 
Criterion of Detection. t 

for example, suppose that the standard deviation, o, of m inatytical 
procedure U 6 yg/litre and that an o of 0.05 is deemed acceptable to that 
the probability of mating a Type I error is set at 5X. The Criterion of 
Detection can then be found from a table of cumulative nonnal probabilities to 
be 1.645 o • 1.645 x 6 wg/11tre • 10 yg/litre. 




Any value observed below 10 yg/litre would be reported as less than the 
Criterion of Detection, since to report such a value otherwise would Increase 
the probability of making a Type I error beyond St. 

Hote that the context of decision Is the analytical result produced by the 
laboratory. A result is obtained and a response made to It. Nothing has been 
said concerning the ability to detect a substance which Is present at a 
specified concentration. 

iCrUerion oi Detection nay be a new term to some. It refers to the minimum 
tnalytlcal result which must be observed before It can be stated that a 
fvbstance has been discerned with an acceptable probability that the statement 
Is true. The terms Detection Limit or Limit of Detection are often used with 
this Maning, but in this Handbook they are reserved for a nore appropriate 
usage. 
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&ice the Criterlw of Detectlor has been set. the probability of making a 
Type II error, P, or Us complement 1-B, the probability of discerning the 
substance when It Is present, can be (Jeteririned for given true situations . 
(The probability 1-B is so«netimes celled the power of the test). 

Cwsider the sarne analytical procedure as above with a Criterion of 
Detection of 10 u9/l^tre. Suppose that the concentration of the sarr.ple 
being analyied Is 10 yg/litre. I.e. the concentration Is equal to the 
Criteria^ of Detection. If, all analytical results below the Criterion of 
Detection were reported as such, then the probability of discerning the 
substance would be 0.5 or 50*. 




Conversely, the probability of naHng a Type II error and failing to 
discerT\ the substance would also be 0.5. From this example It can be seen 
that the probability of discerning a substance when Its concentration is equal 
to the Criteria of Detectlor Is hardly overwhelming. In order for the 
probability of a Type II error to be equal to the probability of a Type I 
error, 6 • a, then the concentration of the sample being analyzed must be 
twice the Criterion of Detection. 
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This concertratior of twice the Criterion of Detection Is the UmU of 
Detection when It has been decided that the risk of Baking a Type II error is 
to be equal to the risk of naklng a Type 1 error. 

The concept of Type II error has been emphasized because It H usually 
ignored. Generally, attention Is paid to the avoidance of Type I error with 
no conslderatiai given to the probability of waking a Type II error. It 
thould also be recognized that when the probability of making a Type I error 
is decreased by selecting a lower «-level. the probability of making a Type 
II error is increased. 

Having, 1t is hoped, made clear the conceptual context In which an 
a-level Is set and the difference between the Criterion of Detection and the 
Limit of Detection. IJC requirements in the reporting of low level data can be 
considered. 

In genera}, only under highlit exceptional circumstances aeed 
there be a concern with avoiding Type I error when report ing 
data for IJC purposes. 

■ There are two reasons why Type I error Is not a concern. First, t.ie IJC 
is not an enforcement agency, and therefore Is not concerned that a single 
datum will lead it Into a false accusation that a substance is present when U 
is not. Second, in virtually all cases data are aggregated for IJC purposes 
in order to provide estimates of loadings and/or concentrations; therefore the 
avoidance of Type I error relates to data sets and not to the individual datum. 

This second point is crucial. Rarely, if ever, will the analytical 
chemist have responsibility for inference from data sets or even be in a 
position to know which data may be combined. Therefore, censoring of results 
to prevent a possible faulty inference being drawn from an individual datum 
represents an unwarrented assumption of responsibility. 

In practice, these ctxisideratltfvs mean that the Criterion of Detectior, may 
be set as low as possible. To state it another way. the o-level may be 

ignored. 
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Od the other bind, when report ijig data for IJC pvzposes 
every effort must be made to arvoii Type II error. 

The reason Is obvious. When results tre reported as 'less than" or "below 
the Criterion of Detection," they are virtually useless for either estimating 
toadings or concentrations. 

In practice, this consideration aeans that If a number can be obtained, it 
is to be reported. 



CODES TO BE USED IN REPORTING LW LEVEL DATA 

At its April 12, 1976 meeting the Data Quality Subcormittee of the Water 
Quality Board passed a resolution that 2 new codes be made available in data 
storage systems for remarks concerning data used 1n IJC reports. The codes 
are T and U. 

The T code has the following meaning: "Value reported is less than 
Criterion of Detection.' The use of this code warns the data user that the 
individual datum with which It is associated does not, in the Judgement of the 
laboratory which did the analysis, differ significantly from 0. 

It should be recognized than an implied significance test which fails to 
reject the null hypothesis that a result does not differ from a standard value 
in no way diminishes the value of the result as an estimate. To illustrate: 
a result of 9 w9 on a test whose o • 6 yg can not be regarded as 
significantly different from for any a-level less than 0.067; however, 1f 
a significance test were made with a " 0.1. then the null hypothesis would 
be rejected and the result deemed significantly different from 0. 

So the result, 9 vg. could be reported as "Below the Criterion of 
tetection" for all a less than 0.067 and could be reported as simply "9 yg" 
for all a greater than 0.067. But however reported, the result of 9 pg 
remains the best estimate of the true value since changing the risk of making 
a Type I error neither augments or diminishes the value of an estimate. 
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It My be added that low level results ire belter estimates, 1n the sense 
of being Bore precise, than higher results since for all analytical tests with 
which «e are acquainted the standard deviation of the test increases with the 
concentration. 

The W code has the following meaning: 'Value observed is less than lowest 
value reportable under T code." This code is used when a positive value is 
not observed or calculated for a result. In these cases the lowest reportable 
value, which is the lowest positive value which Is observable, is reported 
with the W. 

The following example illustrates the use of the codes; 

Suppose that a laboratory has determined that its Criterion of Detection 
for total phosphorus Is 10 yg/litre, and suppose in addition that the 
smallest increment that can be read on the analytical device corresponds to a 
concentration of 2 yg/Htre. Given these conditions, any value observed 
>10 yg/litre would be reported without an accompanying code; any value 
observed >2 yg and <10 yg would be reported with the T code; if no 
instrument response were observed, the result would be reported as 2W. 



REPORTING NEGATIVE RESULTS 

With many analytical procedures there will always be an instrument 
respcnse, so the U code will not apply. In particular, this lack of 
applicability will occur when a result Is obtained through subtraction of a 
blank correction. In this case negative results will often be obtained; in 
fact, if the cwistituent of interest 1$ not present, ore would expect negative 
results to occur as often as positive. 

In order that valid inferences may be uade from surveillance data, it Is 
1iH3ortant that negative results be reported as such. Consider the following 
three different ways of reporting the tame results. The left hand col unr 
gives results In a heavily censored form; the center colunr has negative 
results censored; the right hand coliitir gives the results as obtained. 
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<3 u9 2 pg 2 yg 

<3 -2 

<3 -1 

4 4 4 

3 3 3 

<3 -3 

<3 1 1 

<3 -I 

<3 a 

<3 2 2 

Nothing can be done with the results In the left hand colunr except to 
conclude that -e don't know whether the constUgent Is present or not; the 
sampling and analytical effort have been wasted. 

If the results \t\ the center column were taken at face value, one could 
conclude that the mean concentration was 1.2 ug with a standard error of the 
mean of 0.467 and 95X confidence limits for the mean of 0.14 yg and 2.25 yg. 
Since the confidence limits do not include zero. It would appear that the 
evidence supports the presence of the constituent. 

Analysis of the uncensored results of the right hand colunr gives a mean 
concentration of 0.5 yg. a standard error of the mean of 0.719, and 95J 
confidence limits for the mean of -1.13 yg and 2.13 yg. The correct 
conclusion can be drawn that the evidence Is Insufficient to support the 
presence of the constituent. 

Note that the censored data of the center column distort both the mean and 
the standard error of the data, making the data appear more precise than they 
are. 

Of course any result of or less which Is reported should be reported 
With the T code. 
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